Software Development – Page 17 – The Blog of Colin Mackay

Follow up on what not to develop

Back in May I wrote about a substandard website I attempted to use in an article entitled “What not to Develop”. I also sent the hotel an email at the same time telling them of the failing of their website, however, I never got a response.

When the post went live initially, I got asked on twitter to name and shame the company in question. I suppose publically decrying a company has the effect that if people start doing that then companies will be pressurised in to providing a better service or product. These days I do not to put in a blog post the name of the company in question until I’ve given them a chance to respond to any email I might have sent. I sent the email on 16 May 2009 at 17:21 (BST), I think that’s quite enough time for a response.

I’ve decided to publish some more details so that people can at least learn from the mistake and not repeat them elsewhere. Essentially, this is an extract of the email (slightly reformatted to fit this blog)

Hello,

I tried to book on your website last night and it didn’t work – it advertised a rate to me then refused to book it. I then tried to use your Contact Us page to send you a message and that also broke and said “The web site you are accessing has experienced an unexpected error. Please contact the website administrator. ”

I don’t know who the web site administrator is, but I can guess it is someone employed by TIG Global given this news story: http://www.hospitalitynet.org/news/4036652.search. Personally, if that is the quality they are delivering I wouldn’t use them again as they are not very good and are at best turning away potential customers and at worst exposing you to needless risk.

In order to [help you to] track down the errors I’ve gone back and replicated the initial problem annotating the pages as I go. You will find a number of graphics files attached.

In [the above image] I show the initial details of my availability search. Check in Friday 31st July, check out Sunday 2nd Aug. 1 adult, 0 children.

In [the above image] I show the next page. This was a pop-up, so opened a new window. The details at the top are correct and match what I’d previously entered. The description of the “Weekend Advanced Purchase” sounds perfect “Valid Friday-Sunday throughout 2009”. I see that it is £150 for the “Total price of the stay”. I press the book button.

In [the above image] I show the next page. This was another pop-up, so opened a second window. I now have 3 windows open just for your hotel. (Is this really necessary?). I spot that the number of nights has increased to 3, so I go to change it back to two. I then get an unhelpfully terse error message that says “Minimum stay: 3” [See the next image]

At this point I’m some what irritated by the experience so go hunting for your contact us page. I see that it is a form only without an email address. I fill in the form and when I’m ready I press the “Submit” button. At this point I get an error page back that includes the message “The following information is meant for the website developer for debugging purposes.” You might want to tell those developers that this information is also useful for attackers and they shouldn’t be displaying it to the public. If the developers were any good what they would have done is get the website to log the information internally and display a general message to the user. If they wanted to tie up a user’s experiences with what is in the log then they might also include a randomly generated (say a GUID – globally unique identifier) identifier that is put in the log and displayed so a user can refer to when explaining what problems they were having at the time.

The error message that should have never been displayed is [as follows].

The details in the error page also contain my original complaint. I think I now understand where the American formatting of culture specific information (e.g. dates) is coming from.The company that produced your website was American and in their arrogance just assumed everyone else was just as comfortable using MONTH/DAY/year. I suspect that same arrogance was also responsible for the other failings I’ve pointed out here.

Regards,

Colin.

So, there you are. The hotel is the Southwark Rose Hotel, and their website was produced by TIG Global. (I’ve recently noticed it actually says that at the bottom of the web pages and I need not have searched for relevant press releases!). Incidentally, you can click on any of the graphics to be taken to my Flickr account to see the full sized version.

Tip of the day #13 (String Equality)

When comparing two strings in a case insensitive manner, use:

myFirstString.Equals(mySecondString, StringComparison.InvariantCultureIgnoreCase)

or, if cultural rules are to be ignored completely* then use:

myFirstString.Equals(mySecondString, StringComparison.OrdinalIgnoreCase)

over:

myFirstString.ToLower() == mySecondString.ToLower()

* The invariant culture is actually a non-region specific English language culture. The ordinal comparison is faster than any culture specific comparison as it uses a much simpler comparison algorithm.

Tip of the Day #12 (Hard coded values)

Don’t hard code VAT or other values that can change, even if they don’t change all that often. Additionally, if you really must hard coded values in a program then make it a named constant rather than a literal value so that it can be tracked down by name. That will make it easier on the person having to maintain the program.

Tip of the Day #11 (iMicros)

iMicros is a plug in for FireFox that lets you automate repetitive tasks. This is a fantastic tool for anyone that develops websites as the flow through the site can be recorded and played back quickly and repeatedly. It reduces tester error by taking the mundane parts out of performing user tests. It can also deal with AJAX, has a stopwatch so you can time page loads, and many other things.

Open Source Risk Management

There are some pretty cool things happening in the open source space on the .NET platform these days and new projects are being started every day. However how do you sort the wheat from the chaff? How do you manage risk when using open source?

In this post I hope to address this issue as it relates specifically to open source frameworks that target .NET. Although I am sure the ideas are probably equally applicable to other forms of open source, I don’t have much experience outside of this field so I can’t really comment.

I have a some fairly simple rules when it comes to open source frameworks and perhaps they show me as being somewhat conservative but it works for me. I see that as trying to manage risks as effectively as I can. These rules are:

It must have a “buzz” surrounding it.
It must have reached at least version 1
It must have at the very least prebuilt libraries/assemblies/executables (as needed)

The “Buzz”

The first rule is that there must be a “buzz” surrounding it. By “buzz” what I really mean is that there is a viable community of people who are using the open source software so that if I need support there will be forums or discussion groups that I can join and ask my questions. This community will also have produced good quality documentation, tutorials, articles, blog posts, and so on.

When I start using a some open source framework I want to be able to ramp up quickly and get using it. I want to see that there is MSDN style documentation for when I need to dig into individual methods, properties and the like. I want to see that there are walk-throughs and tutorials so I can see immediately how everything fits together. If there are videos available they need to be good quality because it is really off putting to try and read source code on a video that’s been over-compressed. Podcasts discussing the high level stuff can be useful, but it has to be of the type that I can easily listen to in the car or while walking down the street because I will rarely listen to a podcast while sat at my PC.

A good open source project will also have fairly active support forums with questions being answered from those that understand the technology. It is interesting that lack of “support” is what many people claim as being one of the big risks with open source. However, how many times have you called Microsoft for support? (assuming you are developing on the Microsoft platform) In the last 14 years I’ve phoned them exactly once. Meanwhile, I’ve used newsgroups and support forums more times than I care to remember. The response from forums is usually quite good and well within the length of time I would generally wish.

However, not everything is rosy with forums, open source or otherwise. A couple of years ago I saw a forum dealing with an open source project where the main (possibly only) developer was an arrogant arse in response to any request for help. His standard response was “read the code”. That was it, end of discussion. There didn’t seem to be that many other people using his library so perhaps he put many people off. If this were a commercial operation, that person would have been removed as a support engineer for fear of losing sales. Then again, how many times do you hear of companies with poor after-sales support.

Incidentally, I’ve heard the response “just read the code” or “read the unit tests” from a number of open source advocates. That is one way to turn me away from the project. I really don’t have time to read the code when I’m evaluating a number of frameworks that do similar things. I want to see if the framework will do the job I want it to do. If it doesn’t do what I want it to do I want a quick easy way to short circuit the evaluation.

Don’t get me wrong here. I am not saying I never read the code. It can be extremely useful to read the code when tracking down a bug somewhere and you need to see what the framework is actually doing. What I’m saying is that it should not be the only source of documentation. I will often get around to reading some of the code before fully committing to the open source framework.

Release Version

The second rule is that it must have reached at least version 1. I’m not interested in using betas, CTPs or nightly builds. There are very few things I will use prior to a release. (Curiously as I type this, I’m doing so on a pre-release version of Windows 7. The other thing I’ll use in a pre-release form is Visual Studio – both major products in an already fairly mature state. I’ve been using Windows since 1992 and Visual Studio since 1995.)

What a version number gives above all else is a point where the developers stick their flag in the ground that people can rally around. Without that I could end up having to use various nightly builds. While the nightly builds are great for the testers and developers on a software system, I don’t think they are good for those building upon the system.

For example, if I have an issue and I go to a forum dealing with the framework I can very easily say I’m using, say, Version 2.1 and know that there will be a good number of people also using that version. If however I go along and say I’m using nightly build 518 then I might find that there are very few other people using that specific nightly build and that any potential solutions to my problem by people using other nightly builds may not be able to take into account some weird quirk that only happened in that specific nightly build.

The “standard” response would likely be to download the latest nightly build and use that. However, I see that as adding extra risk into my project. Apart from anything else I would now have to retest all the places where that new nightly build is being used. Sure, unit tests can help at this stage, but integration tests still need to be run, end-to-end testing, user testing and so on.

One of the advantages of open source is its speedy resolution and it may sound that my strategy here is negating that effort. I don’t believe that is so.

When a project has milestones and release goals the development gets to a point where all the features are there and the public interface has solidified although there may still be some bugs. At this point betas are released and a wider audience can evaluate the framework, report bugs, and generally give it a much more thorough testing that the core development team could do. By the time the framework is RTM’ed (although RTW maybe more appropriate in this internet age) the major bugs will have been worked out.

Once a release has been made even more people will start using it, so some of the more esoteric bugs will be found and a service release will likely become available shortly afterwards that rolls up a number of bug fixes. This is still a lot faster than most commercial offerings.

Pre-built Assemblies

The third rule is that the assemblies must already be built in advance. I don’t want the hassle of building these myself. The process is too prone to error in my opinion.

If I build the assemblies then there are risks based on the way my machine may be set up. I’ve seen systems where the build process is quite flake to begin with and I simply don’t want to add to that. I also don’t want to waste lots of time having to faff about with a build process trying to get it to work as it can easily eat up a lot of time.

Incidentally, I may concede this point once The Horn Package Management Project reaches some level of maturity. However, it looks like it may be more useful for those developing complimentary open source projects to keep up with each other rather than someone like me.

Where’s the risk?

As I said at the start, I’m probably fairly conservative when it comes to using open source. I want to see that others are getting benefits and that the major bugs have been ironed out. Yet, I have no real concerns about using open source projects that are clearly maturing or are mature. In that scenario the risk is fairly minimal, and probably a lot less than using an equivalent commercial offering – After all, you still have the source code if the project stops, what do you have if the company fails?

CC-Attribution:

OSI Logo by Ken Coar (June 10th, 2009)

How to get a value from a text box into the database

This question was asked on a forum and I took some time to construct a reasonably lengthy reply so I’m copying it to my blog for a bit of permanence.

I suspect that many of my regular readers will be dismayed at the lack of proper architecture (e.g. layering) but we all had to start somewhere and I suspect that your first programs were not properly layered or structured either. I know mine certainly weren’t. My aim with this was to show how a simple goal can be achieved, what basic things are needed and how to fit it all together by doing the simplest thing that would work (a mantra from the agile world).

Here’s the post (slightly edited to put back some of the original context):

Okay – Let’s step back and show the whole thing from text box to database. NOTE: that this example shows everything in one place. This is generally considered poor practice, but as you are only just starting I’ll not burden you with the principles of layered architecture and the single responsibility principle and so on. (Just be aware they exist and one day you’ll have to learn about them)

So, let’s say you have a form with two text boxes, one for a name, and one for an age. Lets call them NameTB and AgeTB. The user can enter information in these text boxes and press a button that adds them to the database.

First, we need to get the data from the text boxes into a form we can use.

string name = NameTB.Text;
int age = Convert.ToInt32(AgeTB.Text);

Since text boxes only deal with strings we have to convert the string into a number (an Int32 – a 32bit integer) for the age value.

Now, we need to set up the database connection and command in order to insert this. I’ll assume you already have a connections string to your database, I’ve called it myConnectionString for this example.

SqlConnection myConnection = new SqlConnection(myConnectionString);
SqlCommand myCommand = new SqlCommand("INSERT Person(NameField, AgeField) "+
    "VALUES (@nameParam, @ageParam)", myConnection);

I’ve now set up the SQL Command with an insert statement. I’ve assumed there is a table called Person and it has two columns called NameField and AgeField. I’m also going to insert the values via parameters, which I’ve indicated with @nameParam and @ageParam. SQL Server requires that all parameter names start with an @ symbol. Other databases may vary.

myCommand.Parameters.AddWithValue("@nameParam", name);
myCommand.Parameters.AddWithValue("@ageParam", age);

We’ve now added the parameters into the SQL command and we’ve given each parameter the value we got earlier. Finally:

myConnection.Open();
myComment.ExecuteNonQuery();
myConnection.Close();

This opens the connection, runs the INSERT statement and closes the connection again. We’re using ExecuteNonQuery because we don’t expect any results back from SQL Server. If we were expecting data back (e.g. because we were using a SELECT statement) we could use ExecuteReader (for many rows/columns) or ExecuteScalar (for a single value).

This is a very basic example. I’ve not shown any error checking or exception handling. There is also the implicit assumption that all this code resides inside a button click event, which is considered poor practice for anything but a small or throw away application.

Dynamic Objects in C# 4.0

It seems only very recently that I was posting about this wonderful new feature in C# 3.0 called LINQ and its associated language features such as anonymous types, object initialisers and lambda expressions. Soon C#4.0 will be released and it has a host of new goodies to look forward to.

So far I’ve just been dabbling with dynamic types and the dynamic keyword.

C# has been up until this point a purely statically bound language. That meant that if the compiler couldn’t find the method to bind to then it wasn’t going to compile the application. Other languages, such as Ruby, IronPython, Magik and IronSmalltalk are all dynamically (or late) bound where the decision about where a method call ends up is taken at runtime.

Now if you declare an object as being dynamic the compiler will hold off, just as it would do naturally in a dynamic language. The method need not exist until runtime. The binding happens at runtime. If, like me, you rely heavily on intellisense to figure out if you are doing the right thing then you are going to have to get used to not having it. Since the call will be bound at runtime the compiler (and of course intellisense) won’t know whether a binding is valid or not. So, instead of regular intellisense you get a message simply saying “(dyanmic expression) This operation will be resolved at runtime.”

So, how do you get started with dynamic objects? I suppose the simplest example would be an object graph that is built at runtime that would normally have a fairly dynamic structure, say a nice XML file.

In this example, I’m going to take an XElement object (introduced in .NET 3.5) and wrap it in a new object type I’m creating called DynamicElement. This will inherit from DynamicObject (in the System.Dynamic namespace) which provides various bits of functionality that allows late binding.

using System.Dynamic;
using System.Linq;
using System.Xml.Linq;

namespace ColinAngusMackay.DynamicXml
{
    public class DynamicElement : DynamicObject
    {
        private XElement actualElement;

        public DynamicElement(XElement actualElement)
        {
            this.actualElement = actualElement;
        }

        public override bool TryGetMember(GetMemberBinder binder,
            out object result)
        {
            string name = binder.Name;

            var elements = actualElement.Elements(name);

            int numElements = elements.Count();
            if (numElements == 0)
                return base.TryGetMember(binder, out result);
            if (numElements == 1)
            {
                result = new DynamicElement(elements.First());
                return true;
            }

            result = from e in elements select new DynamicElement(e);
            return true;
        }

        public override string ToString()
        {
            return actualElement.Value;
        }
    }
}

The key method in this class is the override of the TryGetMember method. Everytime the runtime needs to resolve a method call it will call this method to work out what it needs to do.

In this example, all that happens is that if the name matches the name of a child element in the XML then that child element is returned. If there are multiple child elements with the same name then an enumerable collection of child elements is returned.

In the event that there is no match the method defers to the base class. If the binding fails then a RuntimeBinderException is thrown.

However, if the binding works you can make complex or ugly calls look much easier. For example. This program using the DynamicElement class to read the contents of an RSS feed:

using System;
using System.Xml.Linq;
using ColinAngusMackay.DynamicXml;

namespace ConsoleRunner
{
    class Program
    {
        static void Main(string[] args)
        {
            XElement xel = XElement.Parse(SampleXml.RssFeed);
            dynamic del = new DynamicElement(xel);

            Console.WriteLine(del.channel.title);
            Console.WriteLine(del.channel.description);

            foreach (dynamic item in del.channel.item)
                Console.WriteLine(item.title);

            Console.ReadLine();
        }
    }
}

The program starts off by reading in some XML into a regular XElement object. It it then wrapped up into the DynamicElement object, del.

The variable del is declared as a dynamic which lets compiler know that calls to its members will be resolved at runtime.

The program then uses the dynamic object to very easily navigate the XML. In this example, I’ve used the XML of the RSS feed for my blog.

The output of the program is:

The barrier to entry for creating and using dynamic objects is quite low. It will be quite interesting to see how this new feature plays out.

Many people like the additional comfort that static binding at compile time provides as it means less things to go wrong at compile time. Advocates of dynamic binding often argue that if you are doing TDD then the tests will ensure that the application is running correctly.

Of course, dynamic binding, like any other language features, is open to abuse. I fully expect to see on forums examples of people who are using it quite wrongly and getting themselves into a terrible pickle as a result. This does not mean that dynamic objects are bad, it just means programmers need to learn how to use the tools they have correctly.

Caveat Programmator: This blog post is based on early preliminary information using Visual Studio 2010 / .NET Framework 4.0 Beta 1

What not to develop

I was recently looking to book a hotel in Southwark in London. I thought I’d found the perfect hotel, it was inexpensive (by London standards) and close to where I would be visiting. They also had availability on an offer for £75 per night, so long as you checked in and out on specific days, which I happened to be doing. It looked perfect.

But then things started to go wrong.

I selected the rate from the availability page and clicked the “Book” button. The next page popped up (it opened a new window) and the details were pre-populated. However, it had changed the number of nights from 2 to 3. I didn’t want 3 nights, so I changed it back to 2 and I got a rather terse message saying “Minimum Stay: 3”. I’m happy to accept that style of message from a compiler, but not from a public facing website.

I went back and repeated the process wondering if I’d somehow clicked on the wrong rate. I double checked everything this time. Date is correct (but in an American format on a .co.uk website), number of people (1), number of nights (2), number of rooms (1), the room description explicitly gives the rules for the stay conditions for the rate. I meet all the conditions that are presented to me. I press “Book” again…

And it has pre-populated everything again and added an extra night on. I don’t want an extra night! Why even present me with a rate that I can’t have because it doesn’t meet my needs.

By this point I’m more than a wee bit frustrated. So I take off to the website’s contact us page. Instead of providing an email address there is a form to fill in. So, I write a description of the issues I was seeing on their site at which point the site fails again. It failed spectacularly badly. If it had taken me to an error page I would have just shrugged my shoulders and gone off elsewhere. But no, it decided to throw up its internals at me. It vomited details of the SQL Statement that failed, stack traces and so on.

It even had the audacity to tell me that “The following information is meant for the website developer for debugging purposes.” It might have well have said “The following information is meant for an attacker so they can destroy our server.”

So, back to my title, what not to develop. There were many failings on this website that I could see. The user experience was poor to start with and it then descended in to abject failure when it vomited its guts up at me.

1. Don’t use pop-up windows; browsers may block them; they cause confusion for some users. Absolutely do not have a pop-up out of a pop-up; it clutters my screen with needless windows.

2. Don’t have a disconnect between the display locale on the site and the TLD. If you have a geographic TLD then display information in a way that consistent with the culture of that location. e.g. Do not display dates in Month/Day/Year format when you are serving pages on a .co.uk domain. If you have customers from overseas and want to localise content for them then offer that ability, but default to your own locale if you don’t know their preference. Some websites try to be clever and will detect based on the IP of the user but even this isn’t 100% accurate. I’m located in Glasgow, but if you use a IP geo location service it shows me in Greater Manchester.

3. If a user has told you their needs do not present rates that do not meet those needs. If you do want to show near alternatives then make it clear that the details entered do not match the rate displayed, but some minor changes will get the user the rate. Put this information at the bottom or in a different colour. Anything that makes it easily distinguishable.

4. Don’t allow a business rule to mismatch the user friendly description. Make sure that the description of the rate actually matches the business rules that will be used to enforce the rate. If you have a rate that is described to the user as from X to Y don’t have the underlying business rules enforce a stay from X to Z. That will just irritate people.

5. Don’t give users terse error messages; it is unpleasant and unfriendly. If a user has made a mistake then gently point it out.

6. Don’t just send data to the database without validating it first. If a user has typed something that is too long for the column in the database for which it is destined then the software controlling the website should never have attempted to send it to the database in the first place.

7. Don’t display information that could be useful to an attacker. Don’t display stack traces, SQL Statements, system generated error messages, code snippets, etc.

Technorati Tags: fail,website,software development,sql,debug,attack,stack trace,business rules

Rant of the day: Learn to frickin' count!

I was in a shop recently and I bought 6 items at £5 each. A total price of £30, even I can manage that mental arithmetic without resorting to a calculator. However, the till decided that the total price was £30.01. For a penny I really can’t be bothered to argue, but it got me thinking about code quality and wondering about what awfulness must be sitting in that system to create such a simple basic mistake.

My colleagues are probably all aware of my views on code quality. I rant daily whenever I see examples on ineptitude by people that are paid money to write code. I read and respond on forums in order to help others learn their craft, or just get unstuck when they accidentally dig themselves in a hole. However, I see on an almost daily basis these days people posting their homework questions with no apparent attempt to at least try to work it out from themselves.

Take this example I found on Code Project a while ago:

I need to know how to do some simple things with arrays please help with any!
1.Find largest or smallest value
2.Count how many times a given value is in the array
3.Count the number of even or odd integers in the array
4.Add up the sum and compute the mean
5.Create another array of the same size containing the same values in reverse order
Thanks!

This is very obviously an exercise from an introductory course on the language they were studying. They just want someone to give them an answer that they can copy and paste. If this is what they are like now, imagine what they will be like years down the road writing commercial software.

I’ve seen lots of evidence over the years of people writing software by copy and pasting examples from the internet without thought of what is actually going on. This results in slow, bloated, inefficient code that is integrated very badly with the rest of the system, hard to read, hard to debug, and is just generally a complete mess.

If you are tempted to copy and paste some code snippet from the internet for your application then stop and think first. Do you actually understand the code? If not, then don’t copy and paste it. If you don’t understand it, how will you debug it?

I would say that if you are tempted to copy and paste from the internet that you create a very small test application first, paste it in to that and learn how it works. Once you understand how it all fits together and how it works you can then write a version that will integrate in to your application.

While you are at it, write some unit tests to go with it. Make sure you test for edge cases, make sure you test for some normal cases too. If you ever get a bug, then add a test that replicates the bug. So if someone suddenly discovers your software things that 5 times 6 equals 30.01 you can add a test for it, fix the bug and redeploy the system. Hopefully, this would have been caught before the public get a chance to see the glaring error and write blog posts about it.

Rant of the day: IDisposable

My colleagues are probably used to the fact that I rant about code quality frequently. I take code quality very seriously. Not because I’m especially expert in it, but because features of basic code quality make it easier for other people to read and maintain the code.

Today’s irritation comes from some code (replicated in a number of classes I might add) that implements IDisposable. It is a fine interface and by implementing it you are telling the rest of the world that you have some stuff that can’t just be left to the garbage collector to clean up. These are things like file streams, database connections, etc. Any type of scarce resource that you want to hand back as soon as you are finished with it rather than leave it up to the garbage collector.

However, I came across this “gem” in some code today where the class, basically a utility class, contained no fields (so it wasn’t holding on to anything at all, let alone anything that might be a scarce resource). Yet, for some reason it implemented IDisposable. What was it going to dispose? What could it dispose?

The answer was in the code:

public void Dispose()
{
    // Nothing to dispose of.
}

Quite!