Text Configuration files and XML

Configuration files are a problem for both operating systems and applications. Where do you keep them, how are they structured?

Traditionally, Unix systems used text files with wildly varying internal structures, and Windows used either binary data or “.ini” text files (in this sense, “binary” is used for anything that you can’t access directly with a simple text editor). More recently, Windows abandoned .ini files in favor of a binary central registry.

The benefits of using a simple text format for configuration files should be obvious: nothing to learn (assuming that you can figure out WHAT to edit), and far more robust – no internal pointers to get screwed up, etc. The downside is that reading these files is slower, but realistically that is almost insignificant. There is also the annoyance of converting any required binary values to and from text, but again, that shouldn’t be anything that really slows down the reading or writing.

However, the ease of editing can be seen as a disadvantage also: if a proprietary tool is necessary for accessing the files, you can impose consistency checks and possibly avoid the introduction of garbage.

I lean toward text files. It is possible to provide validation tools that CAN be used, and even SHOULD be used under ordinary circumstances, while leaving the ability to directly edit raw text when circumstances are not ordinary. A good example of that philosophy is the “visudoers” command (see http://aplawrence.com/Basics/sudo.html ). You should use visudoers ordinarily, but if for some reason you could not, you can edit the /etc/sudoers file directly. Some systems provide special editors for the /etc/passwd file for the same reasons, and of course your normal “editing” of that is through other tools like “useradd” etc.

However, the structure of text configuration files is still a problem. Everything has its own format: /etc/passwd is nothing like /etc/sudoers and neither have any resemblance to /etc/fstab. There’s no common structure. Unix generally ignored this problem, but Windows tried “.ini” files to impose some consistency. Unfortunately, the .ini format doesn’t really cut it, because it’s just too simplistic. There have been some attempts to extend .ini to allow nesting and other features, but most agree that it just doesn’t have legs and isn’t a good solution. Windows didn’t drop .ini files entirely (they couldn’t, simply because of legacy concerns) but they did strongly discourage their use and also provided some integration of .ini into the Registry: http://msdn.microsoft.com/library/default.asp?url=/library/en-us/dnw98bk/html/inifilesversusregistry.asp With Mac OS X, Apple has taken a different approach and uses XML property lists for most configuration data. Using XML provides a consistent data format that is easily hand or program edited.

For example, here’s the start of my ~/Library/Preferences/com.apple.internetconfig.plist:

<?xml version="1.0" encoding="UTF-8"?> <!DOCTYPE plist PUBLIC "-//Apple Computer//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd"> <plist version="1.0"> <dict>

If we look carefully, it’s easy enough to find the home page setting there:

<key>WWWHomePage</key>                         <dict>                                 <key>ic-data</key>                                 <string>http://www.pcunix.com/index.html</string>                         </dict>

and, of course, many other settings and preferences. Every Mac OS X app uses XML property lists too; see http://aplawrence.com/MacOSX/twosafaris.html for an example of those. Apple doesn’t suggest you edit plists directly, of course: there are tools for setting preferences and you should use these whenever possible. But knowing that I can edit these files easily gives me more confidence in my ability to diagnose and fix problems.

Image/Link Rollover in Javascript

Your web site has been up and running for some time and while you are pleased with the results, you can’t help but think that it needs a little bit more “something”.

Maybe a tweak or two to make it stand out a little more. Nothing flashy or too complicated.

Maybe you want to do something with the links, like add some pictures or a “rollover effect”. Well, maybe not all the links; just one or two. There’s a fine line between classy and gaudy.

The problem is: you know nothing about Javascript. To you and a lot of people Javascript looks like ancient Vulcan. And yet there is a solution. Cut and paste comes to mind.

Just head on over to your friendly free Java source, find what you need and cut and paste it into your web page. Sounds simple enough. BUT. As the universe has shown us time and time again…nothing is simple.

Now a couple of hours have passed and you still haven’t found what you were looking for. It seems the Java codes you have been looking at, all deal with muliple images and a humoungus amount of code to place in the “head section” and nowhere does it give you instructions as to how to make the rest of the Java code fit into your web page. Forget about where you want to put it.

Believe it or not, there is a small compact Javascript that you can paste into your web page and put it where you want it. And the added bonus is: it’s a image link.

You will need three pictures in gif, jpg or bmp: your choice. Two of the three pictures must be the same.

Now here’s the Javascript:

<SCRIPT LANGUAGE=”JavaScript”>

Step one: Paste this code into the Head section of your HTML of your document.

<Head>

<script>

{!– Hide Script function move_in(img_name,img_src) { document[img_name].src=img_src; }

function move_out(img_name,img_src) { document[img_name].src=img_src; }

//End Hide Script–}

</script>

</head?>

Step two: Paste this code into the Body of your Html document

<body>

{a href=”anylink.html

OnMouseOver=”move_in(‘image1′,’image_on.gif’)”

OnMouseOut=”move_out(‘image1′,’image_out.gif’)”}

<img src=”image_out.gif”>

<alt=”Move your mouse over here!” name=”image1″>

< /a>

</body>

</html>

Now look down under the “body” tag and you will see: OnMouseOver=”move_in(‘image1′,’image_on.gif’)”. Image_on.gif is the picture you want displayed. Image_out is what will be seen when you roll your mouse over the image.

So far that’s two images. The final image will go into the last peice of code which is “image_out.gif”. This should be your copy image. You can also change the information in the “alt” tag to whatever you like. Place any link you want in the “link” tag.

This little peice of Javascript code makes a great preview picture link for whatever site you want to link it to. Have fun with this. Experiment with different images or just use text and an image. It’s very versatile. I think it works best in tables and should be most effective in getting some attention for you web pages.

From Dell to Intel, companies have started to put up their own blogs as a new way to communicate with their customers. A lot of smaller businesses are trying to follow in their footsteps (with various levels of success). The question is: “Is it really worth it?”

The Good Stuff

1. Getting Personal. Many businesses already publish newsletters for their clients, but blogging takes client communications to a higher plane. The key is interaction. Blogging is a two-way street, usually in an informal atmosphere. With it, one can create a strong personal relationship with customers, encouraging brand loyalty.

2. Reader-friendly. The basic blog format is clean and simple (unless cluttered up by ads). If you use any of the popular platforms such as WordPress, there are hundreds of great-looking free templates you can choose from. And, since people are used to blogs by this time, it’s easy for them to digest your content and navigate through your site.

3. Uncomplicated. You don’t have to be a very technical person to start a blog. Most can be set up in a matter of minutes. Enhancements like polls, podcast widgets and other plug-and-play add-ons are also available.

4. Low-cost. Blog setup can range from free, if you use a specialized blog host such as Blogger, to a few hundred dollars, depending on whether you plan to get your own webhost and use a personalized domain name. Of course, the latter will appear more professional.

5. Blog as portfolio. Aside from featuring the latest announcements from your company, your business blog can include articles on the latest trends in your industry, to show people that you are up-to-date and that your products and services reflect that. You can also post in-depth how-tos, reports, or opinion pieces that can build your credibility as an expert.

Beware Comparing Floating Point Values Can Be Hazardous!

The curse of the .0000000001 and .9999999999!”

Here’s the problem. Say you’re comparing two floating point numbers
a (71.00) and b (71.00) to see if they are the same. The problem is if
you’ve done any calculations to arrive at these numbers they might
actually be stored as 71.00000000001. Now if one of them is stored that
way and the other isn’t and you compare the two to see if they are
equal you’ll get a FALSE as the response, even though they should be
the same.

This isn’t a bug, it’s how floating point comparisons are designed
to work. It makes it so people writing complex software that requires
floating point arithmetic can do what they need to. Unfortunately for
the rest of us that usually are just using floating point numbers to
represent things like currency (dollars in my case) it really makes
life difficult!

PHP Floating Point Comparison Made Easy

Here’s my php function that compares floating point numbers:

function moneycomp($a,$comp,$b,$decimals=2) {	$res = bccomp($a,$b,$decimals); // php function for comparing floating point numbers with a specified level of precision	switch ($comp) {		case ">":			return ($res==1);		case ">=":			return ($res==1 || $res==0);		case "<":			return ($res==-1);		case "<=":			return ($res==-1 || $res==0);		default:		case "==":			return ($res==0);	}}  // example usage: if ($total > $payments) ...if (moneycomp($total,'>',$payments)) ...  // example usage: if ($debt <= $income) ...if (moneycomp($debt,'<=',$income) ...  // example usage: if ($cost == $price) ...if (moneycomp($cost,'==',$price) ...

So that makes life really simple. You just write your comparisons wrapped in the moneycomp() function.

Javascript Floating Point Comparison Made Easy

And here’s my javascript function that compares floating point numbers:

function moneycomp(a,comp,b,decimals) {	if (!decimals)		decimals = 2;	var multiplier = Math.pow(10,decimals);	a = Math.round(a * multiplier); // multiply to do integer comparison instead of floating point	b = Math.round(b * multiplier);	switch (comp) {		case ">":			return (a > b);		case ">=":			return (a >= b);		case "<":			return (a < b);		case "<=":			return (a <= b);		case "==":			return (a == b);	}	return null;}  // example usage: if (total > payments) ...if (moneycomp(total,'>',payments)) ...  // example usage: if (debt <= income) ...if (moneycomp(debt,'<=',income) ...  // example usage: if (cost == price) ...if (moneycomp(cost,'==',price) ...

Resources And Further Reading

• PHP Floating Point Numbers
• PHP Gotchas!
• PHP Manual: bccomp() function
• Javascript Math Functions
• Mozilla Documentation – Javascript 1.5 Math Object

Introduction

Where an application requires certain values at runtime, and where these values may be changed at irregular intervals, it is common practice to hold these on a database record rather than having them hard-coded into any program. This means that should any of these values ever change it is a simple matter of updating the database rather than changing, compiling and releasing individual program modules. This can be a problem, of course, if one of the modules is missed out.

The traditional approach is to have a single control record containing a separate field for each item of data. However, this has the following disadvantages:

• It can result in a very large record structure, which could be a problem as most database engines have a limit to the number of columns (fields) in a table.
• Adding or removing items on the control record requires a change to the physical database structure. All components which reference this structure would then have to be recompiled.
• Any component which updates any control data will have to lock the entire record. This may cause delays to other components which try to update the same record at the same time.

A flexible approach

After encountering those disadvantages on more than one occasion I decided on a different and more flexible approach. Instead of a single record containing many values I have each value on its own record, which therefore results in many records. The trick is to store these values in separate records in the database, but to present them to the user in a single screen as if they were separate values on a single record. How is this possible? Read on and learn.

First, let us start with the database structure:

CREATE TABLE `mnu_control` (  `record_id` varchar(16) NOT NULL default '',  `field_id` varchar(32) NOT NULL default '',  `field_value` varchar(255) default NULL,  PRIMARY KEY  (`record_id`,`field_id`));

Second, build a screen where these records can be displayed and modified:

As you can see this looks like a completely normal screen, which is the whole idea.

Implementation

Although I have previous implemented this design in a different language this article shows my latest implementation in PHP using my own development framework.

Changing the table structure

The first step is to update the internal table definition by replacing the physical database structure with the theoretical structure. The "physical" structure, as exported from the Data Dictionary, is as follows:

    $fieldspec['record_id']                 = array('type' => 'string',                                                    'size' => 16,                                                    'pkey' => 'y',                                                    'required' => 'y',                                                    'uppercase' => 'y');      $fieldspec['field_id']                  = array('type' => 'string',                                                    'size' => 32,                                                    'pkey' => 'y',                                                    'required' => 'y',                                                    'uppercase' => 'y');      $fieldspec['field_value']               = array('type' => 'string',                                                    'size' => 255);

This structure can be replaced at runtime with

the following code:

    function _cm_changeConfig ($where, $fieldarray)    // Change the table configuration for the duration of this instance.    {        // default language code        $fieldspec['default_language']      = array('type' => 'string',                                                    'size' => 5,                                                    'required' => 'y',                                                    'lowercase' => 'y',                                                    'control' => 'dropdown',                                                    'optionlist' => 'language_code');          // how often must the user change his password?        $fieldspec['pswd_change']           = array('type' => 'string',                                                    'size' => 2,                                                    'required' => 'y',                                                    'uppercase' => 'y',                                                    'control' => 'radiogroup',                                                    'optionlist' => 'pswd_change',                                                    'align_hv' => 'vertical');        // change password after 'n' logons        $fieldspec['pswd_count']            = array('type' => 'integer',                                                    'size' => 3,                                                    'unsigned' => 'y');        // change password after 'n' days        $fieldspec['pswd_days']             = array('type' => 'integer',                                                    'size' => 3,                                                    'unsigned' => 'y');        // an invalid password can be tried 'n' times after which the user_id will be disabled        $fieldspec['pswd_retries']          = array('type' => 'integer',                                                    'size' => 3,                                                    'unsigned' => 'y');        // issue a "password will expire in N days/logons" warning        $fieldspec['pswd_warning']          = array('type' => 'integer',                                                    'size' => 3,                                                    'unsigned' => 'y');        // specify the format of user passwords        $fieldspec['pswd_format_minlen']    = array('type' => 'integer',                                                    'size' => 3,                                                    'unsigned' => 'y',                                                    'required' => 'y',                                                    'minvalue' => 1);        $fieldspec['pswd_format_upper']     = array('type' => 'integer',                                                    'size' => 3,                                                    'unsigned' => 'y');        $fieldspec['pswd_format_lower']     = array('type' => 'integer',                                                    'size' => 3,                                                    'unsigned' => 'y');        $fieldspec['pswd_format_digits']    = array('type' => 'integer',                                                    'size' => 3,                                                    'unsigned' => 'y');        // are passwords to be encrypted on the database?        $fieldspec['pswd_encrypt']          = array('type' => 'boolean',                                                    'true' => 'Y',                                                    'false' => 'N');        // are passwords to be visible in the update/enquiry screens?        $fieldspec['pswd_hidden']           = array('type' => 'boolean',                                                    'true' => 'Y',                                                    'false' => 'N');          // define lockout times between which system is unavailable        $fieldspec['shutdown_start']        = array('type' => 'time',                                                    'size' => 5);        $fieldspec['shutdown_end']          = array('type' => 'time',                                                    'size' => 5);        $fieldspec['shutdown_warning']      = array('type' => 'time',                                                    'size' => 5);          $day_names = getLanguageArray('day_names_short');          $fieldspec['shutdown_monday']       = array('type' => 'boolean',                                                    'true' => 'Y',                                                    'false' => 'N',                                                    'control' => 'checkbox',                                                    'label' => $day_names['mon'],                                                    'align_lr' => 'left');          $fieldspec['shutdown_tuesday']      = array('type' => 'boolean',                                                    'true' => 'Y',                                                    'false' => 'N',                                                    'control' => 'checkbox',                                                    'label' => $day_names['tue'],                                                    'align_lr' => 'left');        $fieldspec['shutdown_wednesday']    = array('type' => 'boolean',                                                    'true' => 'Y',                                                    'false' => 'N',                                                    'control' => 'checkbox',                                                    'label' => $day_names['wed'],                                                    'align_lr' => 'left');        $fieldspec['shutdown_thursday']     = array('type' => 'boolean',                                                    'true' => 'Y',                                                    'false' => 'N',                                                    'control' => 'checkbox',                                                    'label' => $day_names['thu'],                                                    'align_lr' => 'left');        $fieldspec['shutdown_friday']       = array('type' => 'boolean',                                                    'true' => 'Y',                                                    'false' => 'N',                                                    'control' => 'checkbox',                                                    'label' => $day_names['fri'],                                                    'align_lr' => 'left');        $fieldspec['shutdown_saturday']     = array('type' => 'boolean',                                                    'true' => 'Y',                                                    'false' => 'N',                                                    'control' => 'checkbox',                                                    'label' => $day_names['sat'],                                                    'align_lr' => 'left');        $fieldspec['shutdown_sunday']       = array('type' => 'boolean',                                                    'true' => 'Y',                                                    'false' => 'N',                                                    'control' => 'checkbox',                                                    'label' => $day_names['sun'],                                                    'align_lr' => 'left');          $this->fieldspec = $fieldspec;          return $fieldarray;      } // _cm_changeConfig 

This amended structure identifies the following:

• What fields need to be displayed on the screen.
• How each field should be displayed (i.e. which HTML control to use).
• How the user input for each field should be validated.

From database to screen

The second step is to read multiple records from the database into an array using the following code:

$fieldarray = $object->getData("record_id='SYSTEM'");

At this point $fieldarray is a nested array – the first level is indexed by row number, and each row contains an associative array of name=value pairs. This looks like the following:

$rowdata => Array(    [0] => Array        (            [record_id] => SYSTEM            [field_id] => DEFAULT_LANGUAGE            [field_value] => en        )      [1] => Array        (            [record_id] => SYSTEM            [field_id] => PSWD_CHANGE            [field_value] => AR        )      [2] => Array        (            [record_id] => SYSTEM            [field_id] => PSWD_COUNT            [field_value] =>         )      [3] => Array        (            [record_id] => SYSTEM            [field_id] => PSWD_DAYS            [field_value] =>         )      [4] => Array        (            [record_id] => SYSTEM            [field_id] => PSWD_ENCRYPT            [field_value] => Y        )      [5] => Array        (            [record_id] => SYSTEM            [field_id] => PSWD_FORMAT_DIGITS            [field_value] =>         )      [6] => Array        (            [record_id] => SYSTEM            [field_id] => PSWD_FORMAT_LOWER            [field_value] =>          )      [7] => Array        (            [record_id] => SYSTEM            [field_id] => PSWD_FORMAT_MINLEN            [field_value] => 4        )      [8] => Array        (            [record_id] => SYSTEM            [field_id] => PSWD_FORMAT_UPPER            [field_value] =>          )      [9] => Array        (            [record_id] => SYSTEM            [field_id] => PSWD_HIDDEN            [field_value] => Y        )      [10] => Array        (            [record_id] => SYSTEM            [field_id] => PSWD_RETRIES            [field_value] => 3        )      [11] => Array        (            [record_id] => SYSTEM            [field_id] => PSWD_WARNING            [field_value] => 5        )      [12] => Array        (            [record_id] => SYSTEM            [field_id] => SHUTDOWN_END            [field_value] =>         )      [13] => Array        (            [record_id] => SYSTEM            [field_id] => SHUTDOWN_FRIDAY            [field_value] =>         )      [14] => Array        (            [record_id] => SYSTEM            [field_id] => SHUTDOWN_MONDAY            [field_value] =>         )      [15] => Array        (            [record_id] => SYSTEM            [field_id] => SHUTDOWN_SATURDAY            [field_value] =>         )      [16] => Array        (            [record_id] => SYSTEM            [field_id] => SHUTDOWN_START            [field_value] =>         )      [17] => Array        (            [record_id] => SYSTEM            [field_id] => SHUTDOWN_SUNDAY            [field_value] =>         )      [18] => Array        (            [record_id] => SYSTEM            [field_id] => SHUTDOWN_THURSDAY            [field_value] =>         )      [19] => Array        (            [record_id] => SYSTEM            [field_id] => SHUTDOWN_TUESDAY            [field_value] =>         )      [20] => Array        (            [record_id] => SYSTEM            [field_id] => SHUTDOWN_WARNING            [field_value] =>         )      [21] => Array        (            [record_id] => SYSTEM            [field_id] => SHUTDOWN_WEDNESDAY            [field_value] =>         ))

This can be changed into a single row of data using the following code:

    function _cm_post_getData ($rowdata, &$where)    // perform custom processing after database record(s) are retrieved.    // NOTE: $where is passed BY REFERENCE so that it may be modified.    {        // turn multiple rows into a single associative array        foreach ($rowdata as $row => $data) {            $fieldarray[0][strtolower($data['field_id'])] = $data['field_value'];        } // foreach          // get list of (virtual) fields in this table        $fieldspec = $this->getFieldSpec();          // insert any missing fields from $fieldspec        foreach ($fieldspec as $fieldname => $spec) {            if (!array_key_exists(strtolower($fieldname), $fieldarray[0])) {                $fieldarray[0][$fieldname] = null;            } // if        } // foreach          return $fieldarray;      } // _cm_post_getData 

The new array looks like the following:

$fieldarray => Array(    [0] => Array        (            [default_language] => en            [pswd_change] => AR            [pswd_count] =>             [pswd_days] =>             [pswd_encrypt] => Y            [pswd_format_digits] =>             [pswd_format_lower] =>             [pswd_format_minlen] => 4            [pswd_format_upper] =>             [pswd_hidden] => Y            [pswd_retries] => 3            [pswd_warning] => 5            [shutdown_end] =>             [shutdown_friday] =>             [shutdown_monday] =>             [shutdown_saturday] =>             [shutdown_start] =>             [shutdown_sunday] =>             [shutdown_thursday] =>             [shutdown_tuesday] =>             [shutdown_warning] =>             [shutdown_wednesday] =>         ))

This data is transferred to an XML document which is transformed into HTML by an XSL stylesheet. The information in the modified structure tells the stylesheet which HTML control to use for each field.

From screen to database

After the user has changed any values he presses the "submit" button to send those changes to the server for processing. Everyone knows that user input should never be trusted, and should be "cleansed" or "filtered" before being written to the database, and this common task can be performed automatically by the framework using the information contained with the modified structure. This will ensure that:

• Any field marked as "required" is not empty.
• All string fields contain strings which do not exceed their maximum size.
• All numeric fields contain numbers which do not exceed their maximum size, or fall below their minimum value.
• All boolean fields contain a value which is either TRUE or FALSE.

After this validation has been performed the data can be written to the database using the following code:

    function _cm_updateSelection($fieldarray, $replace)    // update multiple rows in a single operation.    {        $errors = array();          // set $fieldspec to the database view        $this->fieldspec = $this->getFieldSpec_original();          // get array of fieldnames in the primary key        $pkeynames = $this->getPkeyNames();          // now turn the array of columns into an array of rows        $rowdata = array();        $rownum = 0;        foreach ($updatearray as $fieldname => $fieldvalue) {            $rowdata[$rownum]['record_id']   = 'system';            $rowdata[$rownum]['field_id']    = $fieldname;            $rowdata[$rownum]['field_value'] = $fieldvalue;            // construct 'where' clause from primary key            $where = array2where($rowdata[$rownum], $pkeynames);              // find out if this record currently exists or not            $count = $this->getCount($where);            if ($count == 0) {                // record does not exist, so create it                $rowdata[$rownum] = $this->insertRecord($rowdata[$rownum]);            } else {                // record already exists, so update it                $rowdata[$rownum] = $this->updateRecord($rowdata[$rownum]);            } // if              if (!empty($this->errors)) {                // ignore 'name' and extract 'value' from $this->errors                // as 'name' may not be the same as $fieldname                $errors[$fieldname] = array_shift($this->errors);            } // if            $rownum = $rownum + 1;        } // foreach          $this->errors = $errors;          return $fieldarray;      } // _cm_updateSelection 

The getFieldSpec_original() method is used to replace the modified structure with the original structure. It then steps through the

input array and extracts each field which it then treats as a separate database

row. This row is then inserted or updated, as appropriate.

Summary

This design has the following advantages:

• Values can be added to or removed from the control table without having to

  alter the structure of the table. The structure of the table remains static

  while it is only the contents which are varied.

• Because individual values are held on totally separate records, individual

  values can be locked without affecting other values.

• Because each value is held on a separate record it is easier for the table

  to hold 1000′s of records than it is for the table to contain 1000′s of fields.

• By using a different value for record_id it is possible to use

  the same table for different sets of control data, such as for an individual

  application instead of the whole system.

While websites are great for gaining immediate global exposure (or notoriety, as the case may be), there are times when your product would benefit more from local search engine targeting. For example, you have a pizzeria in Trieste, Italy. A person in Hawaii may accidentally stumble on your site and get a craving for your mouth-watering oven-hot thin crust Four Cheeses pizza… but he can’t order it anyway. It makes much more sense to concentrate your efforts on the people in and around your locality. For domains with a country code like .sg for Singapore and .jp for Japan, it’s automatically done, but if you’re using a neutral domain extension such as .com or .org, it has to be arranged manually. Here’s how to set it up using Google’s geographic targeting.

1. First of all, you must have a Google Account. If you don’t have one, just register for free.
2. Sign up on the Google Webmaster Tools main page.
3. On the dashboard, add the URL of your site. Verify by following the instructions given. If you’ve already done this before, then just select your site URL from the dashboard and proceed to the next step.
4. On the left column, click Tools from the menu.
5. On the Tools page, select Set geographic target.
6. Next, change the setting to “Associate a geographic location with this site”.
7. A dropdown box will appear and you’ll then be able to choose the country/region for your site’s target audience.
8. Save the changes and exit.

That’s it. Your site is targeted for your country. Note that this setting will only be in effect if the person searching chooses the “Search pages from [country]” option on the Google homepage. Otherwise, your site may still be found by users worldwide. Try it and see the results over time. If you want to revert to your previous settings, just log back in to Google and do some quick editing.

Part 3 – “Photoshop to XHTML in 24 Hours”
Part 4 – “Cleaning Up Your XHTML”
Part 5 – “Preloading Images with Javascript and CSS”
Part 6 – “Marking Up is Hard to Do” and “The Anatomy of a WordPress Theme”
Part 7 – “Beginning with PHP for WordPress”
Part 8 – “Putting the Press in WordPress with PHP”
Part 9 – “Marking Up Header.php, Footer.php and Sidebar.php”

So far we’ve made a ton of progress with our theme design. We designed it in Photoshop, we converted to XHTML and then we began the somewhat tedious task of adding PHP . Now that we’ve created index.php, header.php, footer.php and sidebar.php, we can move on to creating the rest of the theme.

In “The Anatomy of a WordPress Theme” we noted that WordPress needs certain files for your theme to function properly. Here’s that list again:

1. 404 Template = 404.php
2. Archive Template = archive.php
3. Archive Index Page = archives.php
4. Comments Template = comments.php
5. Footer Template = footer.php
6. Header Template = header.php
7. Links = links.php
8. Main Template = index.php
9. Page Template = page.php
10. Popup Comments Template = comments-popup.php
11. Post Template = single.php
12. Search Form = searchform.php
13. Search Template = search.php
14. Sidebar Template = sidebar.php
15. Stylesheet = style.css

As you may have noticed, the items listed in bold are the only items we’ve created in our lessons thus far and there is still a lot of work to be done! Anything struck through is optional therefore this lesson won’t cover it but I have linked to pages where you can read about them on your own. While I can’t go into complete detail I will highlight some major points about each and then I’ll give you links to four sites that break the process down in detail and more thoroughly:

single.php

Single.php is the file referenced when someone clicks on the permalink for one of your posts. Thus it is the template for displaying a single post on your blog. On most blogs single.php an index.php are very similar except that in addition to querying <? the content(); ?> we also have to query the comments page. We do this just before we close the loop with:

 	<?php comments_template(); ?> 

page.php

Page.php is the template for any page in your theme that doesn’t have it’s own .php template or that is not a blog entry. This is also the template for the static “Pages” that you create within the WordPress Admin panel. It is usually also just like index.php except that it doesn’t include the loop or the_content().

links.php

Ignores Page content and instead displays your links using get_links_list.

404.php

Where you users land when they follow a bad link on your blog or click through to something that’s been removed. Codex offers some advice….

While you work hard to make sure that every link actually goes to a specific web page on your site, there is always a chance that a link clicked will slam dunk and become a famous 404 ERROR PAGE NOT FOUND. Some errors are avoidable, you should regularly check and double check all your links. Also, if you are deleting a popular but out-of-date post, consider deleting the body of the post, and replacing it with a link referring visitors to the new page.

This concludes “Designing and Coding a WordPress theme from Scratch” . This guide was written as a starting point and a reference guide I can always be contacted at jongos [at] gmail if you are having trouble or need advice. Alternatively I’ve compiled a list of resources that I often reference when creating themes below.

In this tutorial I mentioned a theme that I was developing quite a bit. You can download it along with the PSD used to create it here . Keep in mind that the version you’re downloading is an early beta release. It’s bulky, clunky, not very pretty under the covers and has a mind of it’s own so I’m dubbing it the Rosie O Donnell release. Feel free to give me feedback on it, use it to double check your own code, or modify it to suit your needs.

DOWNLOAD THE AQUA MARINA WORDPRESS THEME

Stepping_Into_Templates
HTML to XHTML
So You Want To Create WordPress Themes
From XHTML/CSS to WordPress
Tamba2′s Guide
Slizing a WordPress Theme

Color Code Converter

This article gives you a color code converter — RGB to hexadecimal, and the other way around.

It is one of the easiest to use color code converters on the Internet.

This one has what many don’t, a color swatch representing the latest converted color code. And not just a swatch, but three swatches so colors can be visually compared with each other.

The demonstration and code download page is at http://willmaster.com/a/26t/pl.pl?art262

Give it a try. If you like it, download the code.

The Color Code Converter can be put on a web page for use by your site visitors.

If you prefer a private installation, you might put it into a popup linked from your personal portal page. (See http://willmaster.com/possibilities/wmptips/tip031014.shtml for a “Personal Portal Page” tip.)

Alternatively, bookmark the demonstration page and use it whenever you need a converter or to compare colors.

The converter has two buttons, one to convert to hexadecimal characters and one to convert to RGB.

There is also a checkbox that, when checked, will convert the codes as you type. It knows which code you’re typing and converts it to the other automatically.

The reason for both methods, manually clicking a button and the auto-convert checkbox, is that some browsers don’t do the auto-convert very well. The conversion is okay, but tabbing from one code field to another can require manual character selection and replacement instead of automatically replacing with whatever is typed. It has to do with the onkeyup() JavaScript command. On some browsers, the command intercepts tab and other non-printable/command keys as if they were normal keyboard printable character keys.

The form fields immediately below the color swatches are hexadecimal color codes for copying and pasting into documents, such as the aforementioned web page source code.

The hexadecimal code is not preceded with a “#” character. If you prefer to have the character for copying with the hexadecimal code, modify the JavaScript as below:

The last function in the demonstration JavaScript is ConvertASection(). Four lines from the bottom of that function’s source code is a line adding the “#” character to the hexadecimal code.

In order to print the “#” with the hexadecimal number in the form fields below the colors swatches, the line needs to be switched with the line immediately above it in the function’s source code. Unfortunately, the line above it is too long to print in this article. But I believe you’ll recognize it even with the ellipsis to indicate missing characters.

So what you do is switch these two lines:

 eval('document.ColorCodeConversionForm.hh ... hgbval + "'"); hgbval = '#' + hgbval;

You’ll end up with:

 hgbval = '#' + hgbval; eval('document.ColorCodeConversionForm.hh ... hgbval + "'");

A live implementation of the converter, with the above edits, is used at one of our affiliate text ad generators, http://willmaster.com/wmaf/AdCodeGenerator.shtml

If you wish to learn from the JavaScript, make a copy and play. Try this and that and see what happens.

Once you have a satisfactory modification, remember to test it on all the different browsers you have access to. IE5 for Mac OS 9 is especially cantankerous when it comes to certain JavaScript functions and constructs. That browser is one reason I rarely use regular expressions in JavaScript. It chokes on the code.

The color code convert can be used to convert from one format to another, to do trial and error until the correct color is displayed in the swatch, and to compare a swatch’s color with those in other swathes.

Create Your Own Darwinian Breeding Grounds

Based on the Darwinian principle of survival of the fittest, genetic programming uses mutation and replication to produce algorithms for creating ever-improving computer programs. In this column, you’ll get to know the genetic algorithm in simple terms. Ted provides Perl implementations for some specific tasks, which you can adapt for generic use. To demonstrate the genetic algorithm, Ted breeds numbers for fitness to a formula, and letters to form English words.

You can run the examples in this article if you have Perl 5.005 or later installed on your system. Preferably, your system should be a recent (2000 or later) mainstream UNIX (Linux, Solaris, BSD) installation, but other types of operating systems may work as well. The examples may work with earlier versions of Perl and UNIX, and other operating systems, but their failure to function should be considered an exercise for the reader to solve.

History

The progress of genetics in the 20th century was rivaled in speed and reach only by the evolution of electronics and computer science. It is fitting that one of the most intriguing algorithms to come about in the 20 th century is the genetic algorithm.

Emerging in the early 1960s, the genetic algorithm (and evolutionary algorithms in general) took a place in computer science between deterministic and non-deterministic algorithms. The genetic algorithm, essentially, is as deterministic as you want to make it, meaning that the user can decide the number of iterations and termination criteria. It mimics Darwinian natural selection, making “fitness” (as determined by a formula applied to an individual) the chief selector of individuals for survival and procreation, together with mutation.

Other evolutionary algorithms attempt to mimic Lamarckian evolution, where behavior as a survival mechanism can be transferred between generations, and there are even evolutionary programs that write themselves for a particular purpose. All of those are beyond the scope of this article.

The main drawback of Perl for genetic algorithms is speed. Because of the computing needs of genetic algorithms, they are more efficiently implemented in C or other low-level pre-compiled languages. The Perl examples shown here are not as fast as their equivalents in C, but they will show you how the genetic algorithm works, and they are fast enough for some problems.

Resources

• Read Ted’s other Perl articles in the “Cultured Perl” series on developerWorks.

• Thanks to Abigail, whose CPAN Sample module exhibits the sample() function I use in both examples. The Sample module and its documentation are wonderful tools for any Perl programmer at Sample .

• Visit CPAN for all the Perl modules you ever wanted.

• Check out Perl.com for Perl information and related resources.

• Visit perldoc.com for Perldoc information online.

• Programming Perl Third Edition” , by Larry Wall, Tom Christiansen, and Jon Orwant (O’Reilly & Associates 2000) is the best guide to Perl today, up-to-date with 5.005 and 5.6.0.

• “Mastering Algorithms with Perl” , by Jon Orwant, Jarkko Hietaniemi, and John Macdonald (O’Reilly & Associates 1999) is a great compendium of algorithms expressed in Perl. Chapter 14, “Probability,” shows how to do weighted and unweighted probability distributions with Perl.

• The Genetic Algorithms FAQ is quite outdated, but it does point to useful suites of genetic algorithm software, both free and commercial.

• Related developerWorks articles by Teodor Zlatanov include:

• One-liners, 101

• Small observations about the big picture

• Browse more Linux resources on developerWorks.

•Browse more Open source resources on developerWorks.

So What is the Genetic Algorithm?

The genetic algorithm is simple enough for anyone to understand, using biological terms taught in high school. Take a population of individuals, each with DNA of their own. Then measure each individual’s fitness (measured as a function applied to the individual’s DNA) and make the individual more likely to procreate if he is more fit. Extremely unfit individuals are killed off. Every survivor gets a chance to procreate (and it is important that procreation is never denied to a survivor, only made less likely if he is less fit). Merging the parents’ DNA, and then applying a random mutation to the merged DNA simulates procreation. The new individual will, in theory, be as fit as the parents, plus or minus a small variance caused by the mutation. The cycle is then repeated.

There are, obviously, many variables that can affect the genetic algorithm, including population size, generations (iterations of the algorithm), merging method, fitness function, how fitness affects procreation chances, and how much mutation happens.

There are some drawbacks to the algorithm, as well. It works best if the fitness function is applied to the DNA as a series of bits. In other words, it’s best if the DNA is a series of binary options, yes or no. Blue eyes? Black eyes? Red hair? Black hair? Merging of parent DNA and subsequent mutation should not allow certain combinations of bits, because the resulting DNA will not be a valid solution to the original problem. Remember that the “DNA” is nothing more than a solution to a mathematical fitness formula. Some values used in that formula may be invalid — for instance, dividing by zero.

In addition, the genetic algorithm is not bound by time. You pick the number of generations. You can define some goal — for instance, “look for an individual with fitness of 0.99999″ and stop there, but then the algorithm may never end because it hasn’t found that individual. That can be a problem if you set unrealistic goals or too few generations. Trial and error, and a good feel for the problem, are the best ways around this issue.

The fitness formula should return a floating point number between 0 and 1. Other ranges can be used, but in my experience floating point numbers work best. You may want a range between 0 and 32767, for instance, if you want the fitness to be a 7-bit integer for optimization.

Of course, delaying optimization until you know it’s needed is a good idea, so you should at least start with a simple fitness formula. The fitness formula is the most-used function in the genetic algorithm (it will be invoked (population size) x (generations times)), so you should make it as simple and as fast as possible.

There are three “good” ways to exit the genetic algorithm. First, you can decide to exit if there is no variety in the DNA pool anymore. This is a tricky test, actually, and a CPAN module for finding the distance of strings might be useful here, as long as you can represent the DNA as a string. Second, you can exit if you have reached a fitness goal. Unless you have a very good understanding of the fitness formula (in which case, you probably don’t need the genetic algorithm anyway), setting a fitness goal can result in either infinite loops, or an individual who is only “good enough.” Third, you can exit the algorithm after a set number of iterations, or “generations.”

In practice, all three ways (or at least the second and third one) are used to control the genetic algorithm. A few test runs, maybe 10 or 20, will give you a good feel for how quickly the algorithm converges, what kind of fitness you can expect.

Genetic Algorithms FAQ

See the Resources section for the link to Genetic Algorithms FAQ. It also points to suites of genetic algorithm software, both free and commercial. See the Algorithm GA that was obtained from the Genetic Algorithms FAQ.

A Simple Example

The code in Listing 1 uses a single byte as DNA (value between 0 and 255, 8 bits). The fitness formula is applied once to every new individual, taking the byte value of the DNA and dividing it by 256. This means that the fitness formula will always return a number between 0 and 255/256, so it can never reach 1. What do you think the fittest DNA will be?

numbers.pl source 

There are several interesting things about Listing 1. Its main loop is at the beginning, and you should understand all the pieces and how they work together on the population (and yet they are separate, so we can reuse them in the next example). You can execute Listing 1 with numbers.pl .

We build the weights array in the select_parents() function by stacking map() on top of grep(). While it could have been written as a loop, the one-line solution is much cleaner that way, and does not slow the program down significantly.

my @weights = map { $_->{fitness} } grep { $_->{survived} } @$population;

The $population array reference is de-referenced. Then, only elements of the array with the “survived” field (set earlier by the survive() function) get through the grep. The survivors are then distilled to their fitness numbers, which get put in their places in the weights array.

The population size was chosen to be 256, because that way it was easy to initialize every individual to a number equal to its index. Feel free to start with different population sizes.

Mutation rates of more than 1% caused the maximum and minimum fitness to fluctuate wildly. The population never stabilized towards high fitness. Low mutation rates cause the population to reach high fitness as a whole much slower. In the end, 1% was about right for our population size.

The breeding selection algorithm picks one parent by looking at the weights – in effect, every individual gets a chance to be a parent, but the number of parent slots is finite. The second parent is picked at random from the parent population. Why? Well, we could have used weights to determine the second parent as well, but this way we ensure every parenting individual has a chance to participate in the breeding process.

The actual breeding is accomplished with a random 8-bit bitmask. We just AND the bitmask to the first parent’s DNA (remember, it’s just a byte) and AND the inverted bitmask to the second parent’s DNA. The effect is that we pick random bits from one parent, and then the rest of the bits come from the other parent.

The mutation is accomplished by AND and OR of individual DNA with random 8-bit bitmasks.

Oh, and by the way, the most fit DNA was 255, of course. You don’t need to wait 100,000 generations. Just hit Ctrl-C when you are done admiring the status line.

Breeding words

For this example, we will make the DNA 32 bits (5 bytes). Each byte will represent a letter or a space. We could have packed more information into each byte, but it would have obscured the purpose of the example. Every byte’s value (0-255, numerically) will be either A through Z (if the value is between 65 and 90, conveniently chosen to match the ASCII set), or a space (if the value is 0 to 64, or 91 to 255).

Note how much of the example is similar to the example in Listing 1. The dictionary of words follows the program.

words.pl source 

The main problem with this example was that DNA lengths greater than 32 bits were hard to manage. I started out trying to do my own bit operations, which were not only unwieldy, but also extremely slow. Then I tried the Math::BigInt package, which was extremely slow for this purpose. You can execute listing 3 with words.pl.

Finally, I settled on the vec() function — it is quite fast, and was the right choice for handling DNA (essentially, the DNA is a bit vector, a built-in Perl data structure). Use “perldoc -f vec” to find out more about the vec() function.

It is possible to end up with 1024 individuals with 0 fitness. That’s why this example has better safeguards against such an “accident” than the first example.

The init_population(), recombine(), and mutate() functions were changed to handle bit vectors instead of bytes.

The dna_to_words() function is somewhat inefficient, but is not invoked often enough to make a difference. The biggest slowdown comes from the fitness() function trying to match all the words in the dictionary, plus all the letters in the alphabet.

Fitness was calculated as: 2 for each letter in the DNA, plus the frequency of that letter in the dictionary, plus 2^N for every dictionary word of length N in the DNA. The dictionary array and letter frequencies hash were obtained only once (using a closure). Feel free to modify the fitness function and the dictionary to breed your own English words. The fitness formula shown is heavily biased towards letters, and takes a while to converge on English words (though “on” and “in” were pretty frequent visitors).

Conclusion

The evolutionary genetic algorithm is a fascinating topic that can hardly be exhausted in a single article. I hope you experiment with the examples and create your own Darwinian breeding grounds. It is especially entertaining to play with the fitness function in the second example, and watch English words appear out of noise.

The techniques shown in the examples range from beginner to advanced level, so try to understand them thoroughly. Often there is room for improvement. The vec() function is especially interesting. It is perfectly suited for long bit vectors such as DNA or other numeric data.

Write your own genetic algorithm implementation. Compare it with mine, and learn from the shortcomings (not necessarily yours, either). Implementing an algorithm is tricky business. There’s a lot you can do wrong, and only a few ways to get it right.

A look at some of the math libraries for Linux

 Programmers generally fall into two groups when it comes to using math. One group doesn’t use floating point much, if at all, and typically needs integers only for the usual mundane purposes like loop control variables, counters, address arithmetic, and other simple calculations. This group’s math needs rarely require the use of anything more exotic than a 32-bit signed integer. They deal with floating point arithmetic only when necessary. And when floating point arithmetic cannot be avoided, they tend to head for the path of least resistance, using whichever FP format is handy or forced on them. This group includes most system programmers, as well as non-scientific and non-financial coders.

The second group includes the financial, scientific, and hobbyist programmers who don’t just crunch numbers but mercilessly grind them until their CPU glows cherry red. They use online handles like "mantissaMan" and "sqrt-neg-one," and tell jokes with scientific notation punch lines. If you’re wondering whether you are in this second group, you probably aren’t.

This sort of thinking let me to write about multiple-precision math (hereafter MPM) libraries. The fact that there’s a seemingly endless list of implementations of MPM libs available on the net for Linux makes this topic valuable to both groups in our little programmer taxonomy.

Ground rules

Before I get into the specifics of various libraries and where to find them, let’s agree on some terminology.

The most widely quoted and used standard for floating point numbers is IEEE-754, which defines several data formats and exactly how various bit patterns in the fields are interpreted. (See Resources later in this column for an online guide to IEEE-754.)

Mantissa and exponent refer to the two parts of binary-format floating point numbers as they are normally represented. The mantissa is the "base" part of the number, and the exponent is the power of ten (usually) the base is raised to. In decimal, the number 5.12e+3 has a mantissa of 5.12 and an exponent of 3.

Precision measures a number’s "size" in terms of the exact number of digits it can represent. This can be deceptive to the floating-point neophyte, however. Don’t think that because an IEEE-754 single precision number can store a maximum number of 3.40292347e+38 that it can represent every possible floating point number up to that value (or down to the negative of that value). An IEEE-754 single has a mantissa of only 23 bits, and you can’t possibly represent 38 decimal digits in only 23 bits. This is typical of binary floating point number representations: their exponents exceed their grasp. But precision doesn’t just refer to floating point values; it’s also applicable to integers.

I can’t stress enough that floating point is one area where you should pay attention to exactly how your data is represented, particularly when dealing with software-only implementations, such as MPM libs. In those cases, you’re using an artificial construct that was simply invented by a programmer, and not something supported in hardware.

Binary vs. decimal

This leads us to the next issue: binary vs. decimal representation of floating point numbers. The most commonly used floating point formats, whether IEEE-754 conforming or otherwise, are based on binary representations, just as integers are. Unfortunately, this leads to the endless discussion in newsgroups about unexpected results from some floating point operations. The explanation is simple — there are many fractional values that can be represented precisely in decimal notation yet have no exact form in binary. This most often surfaces after repeated calculations with one or more such numbers, leading the result to drift from the expected answer. Or it surfaces when you print a result at high precision, and instead of getting a nice round number like 74.00000, you get 73.99998. If you think this behavior is inconvenient in your astronomy or ray tracing program, imagine the headache it causes scientists or financial institutions, where even a small mistake can be catastrophic in more ways than one.

The most obvious solution to this problem is simply to represent floating point values in decimal form, as in the BCD (binary-coded decimal)/"packed decimal" numbers used in mainframes for decades, and countless other custom formats cooked up by inventive programmers. This will solve the representation problem, but usually at a cost in runtime performance, because the math operations are all carried out entirely in software (as in the MPM libraries discussed below), instead of in hardware in the form of a floating point unit (FPU).

By contrast, multiple-precision integers are almost universally represented as binary numbers, because there’s no benefit to going decimal, as there is with floating point values. Keeping the components in binary actually improves performance in most cases, because the most common approach is to represent each multiple-precision integer as an array of smaller integers, each of which is a format natively supported by the hardware.

Some contenders

So, you find that you need to use truly huge integers or floating point values in a Linux program. Where do you turn? In this section I’ll give you a brief overview of some of the more interesting and useful MPM library packages available on the Net. By no means an exhaustive list, or even a "best of the Net" list, this is just a good starting place.

If you find any math libraries for Linux that are worthy of mention, please e-mail me at the address at the start and end of this column, because this is one topic that I will likely return to in future installments.

apfloat
This package includes support for arbitrary precision numbers in four forms: integers, floating point numbers, rational numbers, and complex numbers. The apfloat source code includes two archives: a common, base portion plus one tailored to various C++ compilers (including a generic ANSI C++ version), as well as versions optimized for 386/486 or Pentium systems. While the package’s author says apfloat should be usable with nearly every C++ compiler, it includes a makefile for gcc, which makes building the library under Linux very easy.

The supported math functions include the basic four arithmetic operations plus inverse root, trigonometric functions (normal, arc, hyperbolic, and inverse hyperbolic), power, exponentiation, a function to calculate pi to a desired precision, and more. There is also a stream output operator.

The package includes full source code plus several demonstration and test programs.

apfloat

• Main sites: www.jjj.de/mtommila/apfloat/
• Download page: http://www.jjj.de/mtommila/apfloat/1.51/
• Maximum precision: Limited only by memory (including disk space)
• Language: C++
• Licensing: Freeware, free for non-commercial use
• Documentation: The apfloat manual is a 35-page PostScript file available from the downloads page. It provides a good introduction to the library, as well as several appendices on number theory and algorithms.

gmp
gmp stands for GNU Multiple Precision, and is, of course, the GNU entry in the field. It supports arbitrary precision operations on signed integers, rational numbers, and floating point numbers, and is, of course, quite at home with Linux.

One interesting facet of gmp is the degree to which it has been tailored to various hardware platforms. The manual points out that the package contains "carefully optimized assembly code for these CPUs: DEC Alpha, Amd 29000, HPPA 1.0 and 1.1, Intel Pentium and generic x86, Intel i960, Motorola MC68000, MC68020, MC88100, and MC88110, Motorola/IBM PowerPC, National NS32000, IBM POWER, MIPS R3000, R4000, SPARCv7, SuperSPARC, generic SPARCv8, and DEC VAX. Some optimizations also for ARM, Clipper, IBM ROMP (RT), and Pyramid AP/XP."

Not surprisingly, the manual also points out that gmp places "a general emphasis on speed (as opposed to simplicity or elegance)." This is a fair statement; gmp is very fast, but because it is written in C, it provides a lower level interface than the other MPM libraries mentioned here. In particular, the lack of operator overloading places an additional burden on the programmer and anyone modifying his or her code in the future. The lack of destructors is more problematic, in that it opens the door for the all-too-familiar memory leak problems, because the gmp data types use dynamically allocated memory, which must be freed via explicit calls to _clear functions.

The gmp package is also pretty sparse in terms of provided math functions. It supports numerous variations of the basic functions, plus square roots, I/O, and random numbers, but little else.

gmp

• Main and download page: http://www.swox.com
• Maximum precision: Limited by memory
• Language: C (the main Web page mentions plans to design a C++ wrapper, but there doesn’t seem to be any progress on that front yet)
• Licensing: GPL (GNU public license)
• Documentation: The manual is online in HTML format at http://www.swox.com/gmp/manual/gmp_toc.html , or you can download it in DVI or PostScript format.

If you experiment with gmp, be sure to download copies of the patches from the main site, because they fix some subtle bugs in the support for rational and floating point numbers.

hfloat
hfloat is a floating-point-only library that was developed under Linux using gcc 2.7.x and also includes the author’s fxt-library, a collection of several FFT implementations. hfloat allows you to choose any number from 2 to 65536 for the radix used in number representations, giving it some notable flexibility. The supported operations include square root, n-th power, some trigonometric functions (not as many as apfloat), and logarithms.

hfloat has a few quirks to watch out for, including initialization syntax (described in "Hints and tips" below), and the way it represents a number’s radix and precision. For example, for a base-10 number you use a radix of 10,000, not 10, and you measure precision in LIMBs, which are 16-bit unsigned integers. These aren’t major problems, particularly for programmers working with this sort of package, but I recommend you pay attention to the author’s descriptions and examples of these items in the manual.

The source code for hfloat is also somewhat unusual in that the author has broken it into several compilation units (the base class, one for the "guts", one for functions, etc.). This will be visible to you only if you intend to modify the package, but it adds a bit to the learning curve as you figure out where everything is.

hfloat

• Main and download site: http://www.jjj.de/hfloat/hfloatpage.html
• Maximum precision: Limited only by memory
• Language: C++
• Licensing: GPL (GNU Public License)
• Documentation: The 15-page manual is available as a separate download, in DVI, PostScript, or Acrobat/PDF format. The manual is a bit terse, but it does cover basic usage of the package and its general layout.

Java’s BigInteger and BigDecimal

Java coders are probably familiar with the java.math package and two of its more interesting classes, BigInteger and BigDecimal. These are both arbitrary precision numeric classes, the first being binary, and the second being decimal, as you might have guessed from the names.

The good news is that these classes have been part of the standard Java implementation since JDK 1.1, so they’re "free" to programmers and require no extra work to distribute/install them on a user’s system (assuming Java itself is installed).

The primary drawback of these classes is that they’re pretty minimalist in comparison with some of the MPM libraries. They support the main four arithmetic operations and some other minor manipulations, but that’s it. There are no trigonometric, log, exponentiation, or other features available. As such, they make a good foundation for Java programmers, but they’re not enough to support many scientific or advanced applications.

One other interesting note is that IBM has made its own drop-in replacement for Sun’s BigDecimal class available. You can download it and documentation from IBM’s site (see Resources ). IBM’s and Sun’s BigDecimal classes differ in two areas. IBM improves in a couple of areas on the Sun version, such as making some conversions to less precise data types throw exceptions instead of simply returning bad data. IBM has also added a greater level of control over rounding, but without sacrificing backwards compatibility with existing code.

Java’s BigInteger and BigDecimal

• Main site (Sun): http://java.sun.com
• Main site (IBM): http://www2.hursley.ibm.com/decimal/
• Sun’s documentation for BigInteger: http://java.sun.com/products/jdk/1.1/docs/api/java.math.BigInteger.html
• Sun’s documentation for BigDecimal: http://java.sun.com/products/jdk/1.1/docs/api/java.math.BigDecimal.html
• Maximum precision: Limited by memory
• Language: Java
• Licensing: See packages
• Documentation: Sun and IBM both provide extensive online and downloadable documentation for their Java products. See the Web sites for more detail.

Hints and tips
You could write several sizable chapters or even an entire book on the benefits and pitfalls of MPM libraries, but here are a few things to watch out for as you experiment.

• Always remember that MPM libraries are quirky animals: Every one has its own personality, and you should never assume you can pick one up, slap it into an existing project, and start coding away. At a minimum you should count on experimenting with the library at two levels: a very simple, naive one, in which you just exercise features to gain familiarity with the package, and also a more intense and application-specific one that should reveal the library’s oddities and give you a feel for its performance as you’ll use it. (And performance can be a particularly thorny issue, because in many cases the authors of some of these libraries have lavished attention on some parts and left others with minimal and slower implementations.)
• Consider initialization issues: Virtually all MPM libraries will let you initialize variables in their extended precision formats using as constants either the native environment’s numbers or string values. This makes sense; it would be silly if you couldn’t initialize a multiple-precision variable to anything larger than what a native number will hold. But you can run into trouble with decimal format multiple-precision floating point variables. For example, if you initialize such a variable with the literal constant 0.1, you’ll get a nasty surprise. Your compiler will normally convert 0.1 into a binary format, resulting in a repeating decimal value, which will be used to initialize your variable, so your variable won’t start off containing exactly 0.1. You can avoid this problem by using a string constant of "0.1" instead of a numeric constant. This will get the desired data into your variable and bypass your compiler’s translation efforts.
• Be careful about semantics: Some libraries, like hfloat, do some unusual things with what seems like "normal" statements. The hfloat manual explains that the line hfloat a = 1234; does not initialize the hfloat a to 1234, but instead sets a’s precision to 1234 digits and initializes it to 0. This is probably not what you would expect, and could lead to some spectacular bugs.
• Know the package’s resource requirements: Some libraries require a bit more tender loving care in order to work properly. In some cases you can simply use the library as provided or recompile it, and then drop it into your project as you would any other dynamically linked library. But some libraries require environment variables to be set or other libraries to be present. These details aren’t a concern for your own work, but if you plan to distribute a program that uses one of these libraries, it can quickly turn into more trouble than your users are willing to accept.
• Another aspect of resource requirements is memory usage. Many of the MPM libraries will use all available memory in extreme circumstances, such as dealing with truly huge numbers, but they give you a way to limit the precision of the numbers and the memory used, or to decide when to stage values to disk instead of keeping them in memory. Depending on how widely distributable your application must be, these controls could mean the difference between a successful product rollout and a flop (no pun intended).
• Handling exceptions, overflows, and underflows: This is another area where MPM libraries vary greatly. Some ignore such issues completely, and others spend considerable effort to give you complete control over and information about how math operations are performed, and the results. Some packages use arbitrary precision representations, as opposed to a fixed precision, eliminating overflow/underflow problems.

Hints and tips

You could write several sizable chapters or even an entire book on the benefits and pitfalls of MPM libraries, but here are a few things to watch out for as you experiment.

• Always remember that MPM libraries are quirky animals: Every one has its own personality, and you should never assume you can pick one up, slap it into an existing project, and start coding away. At a minimum you should count on experimenting with the library at two levels: a very simple, naive one, in which you just exercise features to gain familiarity with the package, and also a more intense and application-specific one that should reveal the library’s oddities and give you a feel for its performance as you’ll use it. (And performance can be a particularly thorny issue, because in many cases the authors of some of these libraries have lavished attention on some parts and left others with minimal and slower implementations.)
• Consider initialization issues: Virtually all MPM libraries will let you initialize variables in their extended precision formats using as constants either the native environment’s numbers or string values. This makes sense; it would be silly if you couldn’t initialize a multiple-precision variable to anything larger than what a native number will hold. But you can run into trouble with decimal format multiple-precision floating point variables. For example, if you initialize such a variable with the literal constant 0.1, you’ll get a nasty surprise. Your compiler will normally convert 0.1 into a binary format, resulting in a repeating decimal value, which will be used to initialize your variable, so your variable won’t start off containing exactly 0.1. You can avoid this problem by using a string constant of "0.1" instead of a numeric constant. This will get the desired data into your variable and bypass your compiler’s translation efforts.
• Be careful about semantics: Some libraries, like hfloat, do some unusual things with what seems like "normal" statements. The hfloat manual explains that the line hfloat a = 1234; does not initialize the hfloat a to 1234, but instead sets a’s precision to 1234 digits and initializes it to 0. This is probably not what you would expect, and could lead to some spectacular bugs.
• Know the package’s resource requirements: Some libraries require a bit more tender loving care in order to work properly. In some cases you can simply use the library as provided or recompile it, and then drop it into your project as you would any other dynamically linked library. But some libraries require environment variables to be set or other libraries to be present. These details aren’t a concern for your own work, but if you plan to distribute a program that uses one of these libraries, it can quickly turn into more trouble than your users are willing to accept.
• Another aspect of resource requirements is memory usage. Many of the MPM libraries will use all available memory in extreme circumstances, such as dealing with truly huge numbers, but they give you a way to limit the precision of the numbers and the memory used, or to decide when to stage values to disk instead of keeping them in memory. Depending on how widely distributable your application must be, these controls could mean the difference between a successful product rollout and a flop (no pun intended).
• Handling exceptions, overflows, and underflows: This is another area where MPM libraries vary greatly. Some ignore such issues completely, and others spend considerable effort to give you complete control over and information about how math operations are performed, and the results. Some packages use arbitrary precision representations, as opposed to a fixed precision, eliminating overflow/underflow problems.

Resources