Email without Outlook
Session Number 32
Ted Roche, MCSD, MCSE, VFP MVP
Ted Roche & Associates, LLC
278 Kearsarge Avenue
Contoocook, NH 03229, USA
Voice: 603-746-5670
Fax: call me
Email: tedroche@tedroche.com
Email is quickly become a standard requirement for many FoxPro applications. The technologies underlying our email system are simultaneously mundane and miraculous, as wondrous as the telephone dial tone. FoxPro can send email using any native email client (ShellExecute), using free or inexpensive third-party DLLs, or via automation with Outlook. This session discusses when it is appropriate to use each of those tools, alternatives for other tools, and discusses the issues surrounding sending large email processing, and processing the resultant replies.
Attendees will learn how to...
Sort through the acronyms surrounding email and understand the underlying technologies.
Evaluate their application needs and identify the matching technologies to solve them.
Locate the resources for free or inexpensive email add-on products.
Write the code needed to send and receive email using FoxPro
Skill Level/Prerequisites: A basic understanding of FoxPro and email.
Email is just text. Almost as stunning a revelation as the recipe for Soylent Green, Email is just text. The trick to it is how it is encoded, transported and decoded. While email may appear on your screen in bright colors and with animation, the contents are actually converted to text for transport. Here's how it works:
The mail client (also called "MUA" or mail user agent) encodes the message. A text message is pretty much sent as is. HTML is also just text. Graphics are converted into a text-encoded format using MIME. Binary attachments are also converted to MIME. The entire body of the message has a header attached to it that describes where the message is sent from, who it is addressed to, and how the body in encoded (and optionally, if there are multiple parts, the encoding information is repeated for each part).
The header is not how a message is routed, however. When the client ("MUA") sends the message to a Mail Transport Agent ("MTA") to be routed, the MTA adds a separate document, an envelope, which contains the information on where the message should be sent. This envelope is what is examined by each of the MTAs involved in routing the message. This is how a BCC can be routed to many destinations: the envelope contains the complete list of recipients, but the BCC information is stripped from the header of the message itself.
When the recipient's email client gets the message, it reconstitutes the various parts of the message into the original files as needed, and displays the message.
The trick to email is how all the stuff we get – goofy movies from our brother-in-law, lame animations, colors, graphics and text – is encoded and routed. There are a number of standards involved, and the Internet Engineering Task Force controls them.
The good news is that there are lots of standards in the email business. The bad news is that there are LOTS of standards. And some of them are, and some of them aren't. Here's a quick run-through of the most important ones
The Internet Engineering Task Force (http://www.ietf.org) maintains the standards used by the Internet community to successfully communicate with one another. Most standards are maintained in documents know as Requests For Comment or RFCs. Here are some current standards:
POP3 (http://www.ietf.org/rfc/rfc1939.txt) the Post Office Protocol specifies the interfaces and message exchanges that occur between a client and a mail server. The communications occur on TCP port 110 and allows a "less well endowed client" to contact a machine with a message transport system and collect its mail. This is a simple "maildrop" system, as compared to the more sophisticated client-server interactions of an IMAP system.
SMTP (http://www.ietf.org/rfc/rfc2821.txt) is the Simple Mail Transfer Protocol that specifies the interchange that goes on between a client desiring to send mail and a server. The standard interface is via TCP port 25.
IMAP4 (http://www.ietf.org/rfc/rfc2060.txt ) describes a more powerful mail server interface, where the client can manage folders on the server, synchronize a local mail store with the server's store, and selectively filter what mail is retrieved.
MIME (http://www.ietf.org/rfc/rfc2045.txt through 2049.txt) document the Multipurpose Internet Mail Extensions, the definitions of how non-text elements (RTF, images, attachments) can be included within a text message.
Proprietary standards introduced and supported by only one vendor have been shown to be problematic historically. Without a standards body to debate modifications, a single entity's decision to "upgrade" or "enhance" a standard can break many implementations of the previous interfaces and protocols. For this reason, I recommend avoiding programming to those standards, unless you have some control over the use of the code. For example, if a client really needs features only available through protocol X, you need to explain the risks involved in Company M choosing to modify that standard in a future release of its operating system, office automation package or programming language.
MAPI (http://www.microsoft.com) the Messaging Application Programming Interface was originally designed as a low-level C interface to Microsoft Mail. With each incarnation of Microsoft's new programming paradigms, it has taken on new capabilities via COM and ActiveX ("Active Messaging" and CDO - Collaborative Data Objects).
MHS: (http://www.novell.com) the Message Handling System was Novell's attempt at creating a de facto standard.
VIM (http://www.lotus.com) the Vendor Independent Messaging standard was developed for cc:Mail, and is supported in Notes and Domino, products of IBM.
MUA, MTA: many texts on mail talk about Mail User Agents and Mail Transport Agents. These are primarily abstract entities and don't always map directly to "mail client" and "mail server," but for most purposes of discussion they are close enough. An MUA interacts with a person, and sends and receives mail to a local or remote MTA. An MTA interacts with other MTAs and MUAs.
Now that you've had the tour of the acronyms and technology involved in the email process, it's time to look at sending mail. Since we know that mail is just text, the challenges are how to format and prepare that text, and how to forward it on to the system.
There are a number of command-line tools that will run with Visual FoxPro. An example, BLAT.EXE, is included with the conference materials.
In many cases, a large amount of programming, like Automation of Outlook or Exchange, or adding a new third-party tool to your distribution, is far more involved than is needed to solve the problem In this case, consider a smaller solution. If you simply need to send off an email message, it can be done with native FoxPro code, with user intervention, or by using a small DLL.
Here's a simple two-line VFP program to send mail. Okay, it cheats a little since it is using a FoxPro Foundation Class but, even so, there is very little code to send mail this way:
oShell
= NEWOBJECT("_shellexecute", ;
HOME(0)+"FFC\_environ.vcx")
oShell.shellexecute("mailto:someone@yourcompany.com"
+ ;
"?subject=How To Send Mail" + ;
"&cc=CarbonCopy@YourCompany.com" + ;
"&bcc=BlindCarbonCopy@SomeoneElse.com" + ;
"&body=How can I send mail?")
The first line creates a ShellExecute object, a wrapper around the Windows API ShellExecute call. The second line invokes the API call, using the HTML standard mailto: link to invoke the mail client. This routine does require that you have a mail client on the machine, and it does require user intervention. But if you need to produce an email form for customer feedback, help, support, or error reporting, this routine can do the trick, independent of which mail system the client has installed, as long as there is one.
Updated information: As I discussed in my sessions, there were some challenges in getting the ShellExecute trick to work with the Mozilla mail client on Windows XP. Here are the steps I took to make it work:
Start, Control Panel, Internet Options. Select the Programs tab, and pick “Mozilla Mail” as your choice in the “E-mail” dropdown.
Check that the registry key HKET_LOCAL_MACHINE\SOFTWARE\Clients\Mail is set to “Mozilla”
Check that the key below that one, HKLM\SOFYTWARE\Clients\Mail\Mozilla\open\command is set to: C:\PROGRA~1\MOZILLA.ORG\MOZILLA\MOZILLA.EXE -mail
The version of Mozilla I tested this on, version 1.2b, Build 2002101612, is a beta version, and has a known bug that invoking the email composer this way also launches a blank Mozilla browser window. Hopefully, this will be fixed in the final shipping version, but it is only a cosmetic annoyance at this point.
BLAT is a public domain implementation of the simplest form of sending email directly via SMTP. It is synchronous, which means that your program will hang until the mail has been sent (on my machine, a 20k message took about 7 seven seconds with the executable and 3-5 seconds with the DLL). BLAT.EXE is a command-line version, more suited for batch file execution, while the BLAT.DLL is easily implemented for VFP using the DECLARE – DLL command. See the web site http://www.blat.com for documentation and links to the latest DLL.
Listing 3 Using the BLAT DLL to send email
LOCAL lcMessage as String, ;
liResult as Integer && integer result code
DECLARE Integer Send IN BLAT.DLL string cMessage
lcMessage = "Message.txt -t tedroche@tedroche.com"
liResult = Send(lcMessage)
? liResult
One of the issues with free products such as BLAT is that they are synchronous – waiting for the server to respond before continuing. In many cases, especially the situation with large mailings, you would prefer to continue, asynchronously, in order to get the best throughput. The wwIPStuff.DLL and class library, built by Rick Strahl of West Wind (http://www.west-wind.com) solves this with a SendAsync() function. Listing 4 shows a sample of sending mail using wwIPStuff.
Listing 4: Using wwIPStuff to send mail
loIPStuff = NEWOBJECT("wwIPStuff","wwipstuff.vcx")
loIPStuff.cMailServer="mail.yourserver.com"
loIpStuff.cSenderemail="user@yourdomain.com"
loIPStuff.cSenderName = "Your Name"
loIPStuff.cRecipient = "user@targetdomain.com"
loIPStuff.cMessage = "One if by land, and two if by sea,..."
? loIPStuff.SendMail()
Sending email is not the specialty of Microsoft's premiere desktop database development system. If your situation requires very high volume, complex processing, or multi-threaded high-speed performance, look for a third-party product that would meet these needs.
Processing incoming email is a little trickier than sending outgoing mail. Outgoing mail is simply text, and you send it, and it goes, or fails. Processing incoming email involves engaging in a dialog with the mail server, determining if the mail should or shouldn't be retrieved.
West Wind Client Tools also supplies DLLs, class libraries and documentation to access a POP3 mailbox. The process for reading mail is a little more time-consuming: you set the properties you need to connect and authenticate to the server, connect, iterate through the messages, perhaps just retrieving headers until you locate the message of interest, download the message, and
*===============================================================
* Program: DEMOPOP3.PRG
* Purpose: Demos the ww Pop3 class
* Author: Rick Strahl, adapted by Ted Roche
*===============================================================
Do load_wwipstuff.prg
loPop = CREATEOBJECT("wwPOP3")
loPop.cMailserver="mail.yourserver.com"
loPop.cUsername="YourUserName"
loPop.Cpassword = INPUTBOX("Enter your password:","Mail Server Password")
loPop.lLogSession = .F.
loPop.nTimeout = 15 && Seconds
loPop.nBufferSize = 1024
? "Connecting..."
?? loPop.Connect()
? "MessageCount: "
?? loPop.nMessageCount
ltTimeStart = DATETIME()
FOR x = 1 TO loPop.nMessageCount
loMsg = loPop.GetMessage(x)
IF ISNULL(loMsg)
? loPop.cErrorMsg
File2Var("pop3.log",loPop.oIP.GetLog())
RETURN
ENDIF
? "To: " + loMsg.cTo
? "Date: " + loMsg.cDate
? "Subject: " + loMsg.cSubject
? REPLICATE("-",60)
* loPop.ParseMultiPartMessage(loMsg)
ENDFOR
? "Elapsed: "
?? DATETIME() - ltTimeStart
loPop.Quit()
Rather than having to become the dependable pipeline for email processing, a better assignment for Visual FoxPro is to deal with the output of an email system. Visual FoxPro is not multi-threaded and cannot always handle the throughput and event firing that would be caused by mail flowing through an organization. Most mail servers have options to create logs or text files of incoming messages, either through an event sink or via a macro language construct. Consider setting VFP up as a daemon checking for the existence of such logs, to allow it to process the email at its rates.
The Internet Engineering Task Force (http://www.ietf.org) develops, maintains and supports the underlying protocols that let the Internet communicate. Visit their site for lots of information on the standards (RFC) process, and for actual copies of the standards documents. The MailTo: URI is an example of an RFC. http://www.ietf.org/rfc/rfc2368.txt specifies that it can be any set of key-value pairs, but "The creator of a mailto URL cannot expect the resolver of a URL to understand more than the "subject" and "body" headers."
D. J. Bernstein is one of the authorities on Internet mail, and he shares his knowledge at http://cr.yp.to/mail.html, http://cr.yp.to/im.html, http://cr.yp.to/smtp.html and http://cr.yp.to/immhf.html. He is also the author of qmail, an email server with an excellent reputation for performance and security.
The Appendix, "How Internet Mail Works" to "The qmail Handbook" by Dave Sill, APress, ISBN 1-893115-40-2, is worth the price of the book alone, and was a great help in understanding Internet mail.
While portions of the rant are a bit extreme, I think that "Why HTML email is evil" at http://www.georgedillon.com/web/html_email_is_evil.shtml has some good points, too.
A great description of how to read mail headers, how they are created, and what transaction happen over the course of an email transmission can be found at http://www.stopspam.org/email/headers/headers.html.
Everyone's needs for a customer solution will vary somewhat, depending on the architecture, volume and type of mail you will need to support. Listed here are some vendors I have used or heard mentioned in discussions. Most should work with FoxPro solutions.
ChilkatMail: free ActiveX control http://www.chilkatsoft.com
A search on Google for with the keywords [command-line Windows "send mail"] gave hundreds of hits, including these:
http://www.totalshareware.com/asp/list_view.asp?catid=99
A number of languages, such as Perl, have email built in.
Microsoft offers CDO, the Collaboration Data Objects, as their latest COM interface into MAPI. However, CDO is only installed by default on the more recent operation systems, and it is dependent on the associated MAPI facilities being available. In addition, the latest security patches will display warning dialogs when CDO is used, since it can be invoked from scripting engines.
Rick Strahl’s West Wind Client Tools (formerly wwIPStuff) is available from the Downloads section of http://www.west-wind.com - $199 shareware, or included with West Wind’s WebConnect. wwIPStuff 4.0 introduced POP3 features into a DLL that already includes HTTP, SMTP, FTP and many other great functions, along with a VFP class wrapper, VFP example code, and a help file.
Redei Enterprises offers REMail, a DLL with RAS, FTP, SMTP and POP capabilities. Examples in VFP are included. Free download for evaluation, $129 to register.
Communications Horizons, the folks who make NetLib, have a TCP/IP Toolkit for Visual FoxPro that includes SMTP and POP http://www.netlib.com/ (look for the "Legacy Products" and "Netlib for Clipper and FoxPro" links to find the Toolkit.)
Marshallsoft offers the SMTP/POP3 Email Engine Library for Visual FoxPro (SEE4FP) as shareware, with a $109 fee for purchase and registration, royalty-free distribution, and free updates for a year. The package support SMTP and POP3 functionality, and includes examples in Visual FoxPro.
Self-billed as "the ultimate command-line mailer", Postie (http://www.infradig.com) is free for personal use, and available with a variety of commercial and redistribution license options. Available as an ActiveX control as well. "Supports NNTP, SMTP, ESMTP, MIME, BASE64 and UUENCODE for posting, and POP3, IMAP4, HTTP protocols for receiving." Also supports SSL/TLS and IPv6. If you know of any other acronyms, I'll bet they'll support them, too!
Find links to the BLAT executable and DLL at http://www.interlog.com/~tcharron/blat.html and http://www.geocities.com/toby_korn/blat/, respectively.
TCMail (http://www.terminalc.com) is another simple, command-line based email sender.
Quicksoft http://www.quiksoft.com/ has a product, SMTP Express, suitable for large email processing, with supports for multiple threaded queues for greatest throughput.
ASPMail offers basic functionality for free, and premium features at a reasonable price (http://www.aspemail.com/)
PeterEastwood.com offers a free POP3 ActiveX control, written in Visual Basic 6
Miguel Garrido offers a POP3 DLL in his "Discontinued software" link at http://www.dd-soft.net/. Samples in VB and VC++, but pretty easy to translate.
Ted Roche consults, teaches and develops LAN, client-server and Web applications using Microsoft Visual FoxPro, Microsoft Visual SourceSafe, SQL Server, and other best-of- breed tools. He is a principal in Ted Roche & Associates, LLC, http://www.tedroche.com/, based in New Hampshire, where the company works with clients nationwide as mentors, teachers and fellow developers. Ted is author of "Essential SourceSafe" (Hentzenwerke Publishing), co-author of the award-winning "Hacker’s Guide to Visual FoxPro" series, a contributor to five other FoxPro books and author of numerous magazine articles. Ted is a Microsoft Certified Solution Developer, Microsoft Certified Systems Engineer, and Microsoft Support Most Valuable Professional. Contact Ted at mailto:tedroche@tedroche.com.