Automatically replacing an image on an HTML page when it is not found.

The project I’m working on has just moved the image hosting to Amazon S3. Previously what happened was that we had a big folder full of images that had been uploaded from our users and that if the site needed to render an image it would check the directory for the image it needed, if it didn’t have it, it would look for the original and then resize and render that (storing the resized version in the folder so it can be found the next time). If the original couldn’t be found either it displayed a replacement image in place that was basically an image that said “There is no image available.”

That worked well enough with a small number of users but it really didn’t scale well.

Now that we’ve moved the hosting to Amazon S3 we create all the image sizes needed at the time they are initially uploaded. If we need a new size we have a tool that will go and create all the resized versions for us. The only issue that remains is that some images don’t exist for various reasons. Much of the legacy data came from systems that were installed on people’s desktops and the image data simply never got sync’ed to the central server properly.

But there is a way around this on the browser. The img tag can have an onerror attribute applied, which can then call a function which replaces the image src with a dummy image that contains the message for when there is no image.

For example:

<div>
  <img src="error.jpg" onerror="replaceImage(this, 'replacement.jpg');" title="This image is replaced on an error"/>
</div>

<script type="text/javascript">
  function replaceImage(image, replacementUrl){
    image.removeAttribute("onerror");
    image.src=replacementUrl;
  }
</script>

Although this looks a little ugly (putting in lots of onerror attributes on images) there is a lot less code to be written. When trying to achieve the same results in jQuery I eventually gave up. That’s not to say that it can’t be done, just that for pragmatic reasons I didn’t pursue it as I was spending too much time trying to get it to work.

The function does two things, first it removes the onerror because if the replacementUrl is also broken it will just recurse the call to the error handler and the browser will just slow right down. Second, it performs the actual replacement.

To see it in action there is an example page to demonstrate it.

I also tried to create a jQuery based solution to fit in with everything else. However, there were a couple of problems with a jQuery solution that were less than ideal.

  • You can’t attach an error event to the images because by the time you have done so the error event will be long past. You have to loop around all the images initially to find out which didn’t load before jQuery got a chance to get going.
  • For images that are added to the page by jQuery itself the .on does not work because delegated events, which allow you to create event handlers on elements before they are created, need the events to bubble up to a parent that did exist at the point the event handler was attached. The error event, among a small set of other events, does not bubble up. And if you attach it directly to the newly created element on the page then it will likely be too late, especially on a fast connection, as it will have already fired off the error event. You could do the same as before and check manually to see if the image loaded or not – but then the code is getting rather unwieldy and unmanageable.

In the end I found that the small bit of code that is called from the onerror attribute on each img element that needed it was more compact and didn’t require lots of extra lines of code to ensure that all the errors were corrected in the case that jQuery just didn’t get there in time.

Finally, if anyone has a solution in jQuery that does not require cluttering up the HTML, I’d like to see it

Tip of the day: Setting a default stylesheet for Chrome

I often type up HTML documents in a text editor and view them directly in my browser to review then. However I don’t like the default style, I don’t find it is very readable. It is too plain and dull. What I really want is to provide a style sheet that make the document styled more to my taste, yet I don’t want to have to litter my HTML files with uncessesary mark up linking to stylesheets.

Luckily Chrome has a custom.css file, which is empty by default, which you can use to provide your own CSS that is used when rendering documents and no other style information is available.

On windows this Custom.css file is located at: C:\Users\<username>\AppData\Local\Google\Chrome\User Data\Default\User StyleSheets\Custom.css

I’ve created my own style sheet, which can be found as a Gist on GitHub: Custom.css.

This is great. My plain HTML documents that I use for writing up blog posts and the like now look a lot better and without me having to put in references to CSS. It also means that the file can contain just the HTML body (no <head> or <html> tags) so I can lazily perform a Ctrl+A, Ctrl+C on the document and then drop it in to my blog.

There is one slight problem with all of this… Depending on how far you go with the styling, a little light play with fonts is okay, but some things like setting borders, margins, colours and widths can mess up existing sites because they don’t expect the base style sheet to contain those elements. But then again, maybe that’s a good thing if you are a front end web developer. It ensures that you don’t make assumptions that may not be valid.

Xander.PasswordValidator – WordListRegExBuilder

When word lists are processed a regular expression is built in order to quickly traverse the lists. The regular expression is built using a number of builders which create various parts of the regular expression. One for checking the password itself, and another for testing the password against the list but modified by adding a numeric suffix. You can add your own by creating your own class derived from WordListRegExBuilder and then adding it to the WordListProcessOptions.

The WordListRegExBuilder is an abstract base class and demands that the GetRegularExpression method is implemented in any concrete derived class. It also has a method called RegExEncode which takes a string and encodes it for use in a regular expression, escaping out all the special symbols used by the regular expression engine.

For example, say you want to validate the password against the list, but check also for a numeric prefix you can create a class to build that part of the regular expression. That class would look something like this:

using Xander.PasswordValidator;
namespace Xander.Demo.PasswordValidator.Web.Mvc3.Helpers
{
  public class NumericPrefixBuilder : WordListRegExBuilder
  {
    public override string GetRegularExpression(string password)
    {
      return "[0-9]" + RegExEncode(password);
    }
  }
}

And to use this in the validator, add it to the settings like this:

var settings = new PasswordValidationSettings();
settings.WordListProcessOptions.CustomBuilders.Add(typeof(NumericPrefixBuilder));

The Validator will create a new instance of your class and run the GetRegularExpression method. It will then incorporate that in to the regular expression that it is building and test word lists using it.

Xander.PasswordValidatator – Validation Handlers

If the password validator does not have the validation rules that you need for your project then it is easily extendable. You can create your own ValidationHander derived classes and set add them via the PasswordValdiationSettings class.

ValidationHandler

The ValidationHandler class is an abstract base class which is extended in the Xander.PasswordValidator framework itself to provide the various built in validation routines. (You can see examples of some of them here on GitHub)

You can create your own by simply creating a class and setting Xander.PasswordValidator.ValidationHandler as the base class and overriding the Validate() method.

The Validate method simply accepts the password as the parameter and returns a Boolean. Return true to indicate that the password passes the validation, false if it fails the validation.

For example, here is a very simple validator that rejects passwords that look like dates:

using System;
using Xander.PasswordValidator;

namespace Demo.ValidationHandlers
{
  public class NoDatesValidationHandler : ValidationHandler
  {
    public override bool Validate(string password)
    {
      DateTime date;
      var parseResult = DateTime.TryParse(password, out date);
      return !parseResult;
    }
  }
}

To set this up so that the validator calls it, it needs to be added as part of the settings. You pass in the type of the handler. The Validation framework will create an instance of the handler for you, if it needs it. If validation fails before it gets a chance to run your validator then your validator will not run.

var settings = new PasswordValidationSettings();
settings.CustomValidators.Add(typeof(NoDatesValidationHandler));

CustomValidationHandler<TData>

This derives from ValidationHandler and is used when you need to pass some additional data or objects into your validation handler for it to work properly.

The Validate method works as before, except you now have access to an additional property from the base class that contains your custom data, called CustomData. CustomData is the object passed in through the settings.

To pass in the data through the settings you use the CustomSettings property on the PasswordValidationSettings class. For example:

settings = new PasswordValidationSettings();
settings.MinimumPasswordLength = 6;
settings.CustomValidators.Add(typeof(PasswordHistoryValidationHandler));
settings.CustomSettings.Add(typeof(PasswordHistoryValidationHandler), new Repository());

The key passed into CustomSettings is a type that refers to the ValidationHandler type that the settings are to be sent to.

The ValidationHandler looks something like this:

using System.Linq;
using System.Web;
using Xander.PasswordValidator;

namespace Demo.ValidationHandlers
{
  public class PasswordHistoryValidationHandler : CustomValidationHandler
  {
    public override bool Validate(string password)
    {
      var user = HttpContext.Current.User;
      var history = CustomData.GetPasswordHistory(user.Identity.Name);
      return !history.Any(h => string.Compare(password, h, true) == 0);
    }
  }
}

In the above example, the settings pass in a repository which is passed on to the ValidationHandler when the Validator is run. The repository is used to get a history of passwords (It is a dummy repository in this example – In real life you should never have access to the plain text passwords like this) and the history can be checked against the current password to ensure that it does not match any of the historical passwords.

There is a slightly updated set of assemblies for this as I made some changes to way the CustomValidationHandler works: PasswordValidator.0.2.0.0.zip

Xander.PasswordValidator – The config file

Earlier in this series I introduced the config file, but I didn’t say much about it other that show an example. In this post I’ll go in to more detail.

Defining the config section

To define the section:

<configSections>
<!-- Set up other config sections here—>
   <sectionGroup name="passwordValidation">
      <section name="rules" type="Xander.PasswordValidator.Config.PasswordValidationSection, Xander.PasswordValidator, Version=0.1.0.0, Culture=neutral, PublicKeyToken=fe72000dffcf195f" allowLocation="true" allowDefinition="Everywhere"/>
   </sectionGroup> </configSections>

This defines the configuration section will will appear later in the config file.

An example of the config section itself:

<!-- The configuration section that describes the configuration for the password validation -->
<passwordValidation>
   <rules minimumPasswordLength="6" needsNumber="false" needsLetter="false" needsSymbol="false">
     <wordListProcessOptions checkForNumberSuffix="true" checkForDoubledUpWord="true" checkForReversedWord="true" />
     <standardWordLists>
       <add value="FemaleNames"/>
       <add value="MaleNames"/>
       <add value="MostCommon500Passwords"/>
       <add value="Surnames"/>
     </standardWordLists>
     <customWordLists>
       <add file="WordLists/MyCustomWordList.txt" />
       <add file="WordLists/MyOtherCustomWordList.txt" />
     </customWordLists>
   </rules> </passwordValidation>

The rules

The rules section defines the actual rules by which the passwords will be validated.

<rules minimumPasswordLength="13" needsNumber="true" needsLetter="true" needsSymbol="true">
  • minimumPasswordLength: a positive integer that defines the minimum number of characters needed for a valid password. It is optional and if missing will default to 8.
  • needsNumber: Boolean that indicates whether the password needs a number in it. It is optional and if missing will default to true.
  • needsLetter: Boolean that indicates whether the password needs a letter in it. It is optional and if missing will default to true.
  • needsSymbol: Boolean that indicates whether the password needs a symbol in it. It is optional and if missing will default to false.

Rules can have a number of child elements also.

  • wordListProcessOptions: A set of options for how the word lists are processed
  • standardWordLists: A collection of built in word lists to use to check the password against.
  • customWordLists: A collection of custom word lists to use to check the password against.

The word list process options

By default, checking the password against the word lists only checks to see if the password is in a word list. These are additional options for checking against the word lists.

<wordListProcessOptions checkForNumberSuffix="true" checkForDoubledUpWord="true" checkForReversedWord="true" />
  • checkForNumberSuffix: Indicates whether the password should be checked to see if it is simply in the word list with an additional digit appended. This is optional, and by default is false.
  • checkForDoubledUpWord: Indicates whether the password should be checked to see if it is the same sequence repeated over again, and if it is to see if the first half is in the word list. This is optional and the default value is false.
  • checkForReversedWord: Indicates the a reversed form of the password should be checked to see if it in the word list. This is optional and the default value is false.

Standard word lists

This element is a container for a collection of standard word list items.

     <standardwordlists>
       <add value="FemaleNames" />
       <add value="MaleNames" />
       <add value="MostCommon500Passwords" />
       <add value="Surnames" />
     </standardwordlists>

The valid words list are:

Custom word lists

This element is a container for a collection of file paths to plain text files that contain custom word lists to check against. A word list file is simply a plain text file with one word per line.

     <customWordLists>
       <add file="WordLists/MyCustomWordList.txt" />
       <add file="WordLists/MyOtherCustomWordList.txt" />
     </customWordLists>

The paths are relative to the working directory of the application in which the password validator is operating. In an ASP.NET web application the paths should be prefixed with the ~ to ensure they are correctly mapped on the server relative to the root of the web application.

Xander.PasswordValidator – In a Web Application

In the last post I introduced Xander.PasswordValidator and showed the basics of how to configure it. In this post I’m going to show the PasswordValidationAttribute and how you can use it in your ASP.NET MVC application.

PasswordValidation attribute

At its simplest, all you need to do is to decorate a property in your model with the PasswordValidationAttribute, like this:

  public class SomeModel
  {
     [PasswordValidation]
     public string Password { get; set; }






   
    // Other stuff goes here
  }

That will validate the password based on the settings in the config file, which I discussed briefly in my previous post, and I’ll go into more detail later.

Registering the Password Validator

In order for the file paths to custom word lists to be resolved correctly in a web application you need to register the validator in the Application_Start() method in your web application’s HttpApplication derived class. (Or anywhere before first use).

For example, the Application_Start() method may look like this:

protected void Application_Start()
{
   PasswordValidatorRegistration.Register(); // Register password validator
   AreaRegistration.RegisterAllAreas();
   RegisterGlobalFilters(GlobalFilters.Filters);
   RegisterRoutes(RouteTable.Routes); }

Validating settings from code

As the settings can get quite complex they cannot be set directly in the attribute that you use to decorate the model. Instead they can be set elsewhere and referenced in the attribute.

The settings can be configured as normal then added to the PasswordValidationSettingsCache. For example:

var settings = new PasswordValidationSettings();
settings.NeedsNumber = true;
settings.NeedsSymbol = true;
settings.MinimumPasswordLength = 6;
settings.StandardWordLists.Add(StandardWordList.FemaleNames);
settings.StandardWordLists.Add(StandardWordList.MaleNames);
settings.StandardWordLists.Add(StandardWordList.Surnames);
settings.StandardWordLists.Add(StandardWordList.MostCommon500Passwords);
PasswordValidationSettingsCache.Add("StandardRules", settings);

This code would typically be placed in the Application_Start() method, after registering the password validator.

The important line is the last one. It adds the setting tot he cache with the name “StandardRules”. That can then be references in the attribute later. Like this:

public class MyModel
{
  [PasswordValidation("StandardRules")]
  public string Password { get; set; } 
}

The PasswordValidationAttribute references the entry in the cache, which is then retrieved to perform the validation.

Xander.PasswordValidator – A Simple Demonstration

I recently introduced the Xander.PasswordValidator project I’m working on in a previous blog post. In this post I intend to demonstrate some of the basics of how to use it.

Validator

At the most core is the Validator class. It performs the validation of the password and returns a value to the caller to let them know if the validation passed or failed.

The validator can take settings set by the caller, or it can find settings in the application’s configuration file.

Here is a simple example of it working:

var settings = new PasswordValidationSettings();
settings.MinimumPasswordLength = 6;
settings.NeedsLetter = true;
settings.NeedsNumber = true;
settings.StandardWordLists.Add(StandardWordList.MostCommon500Passwords);
var validator = new Validator(settings);
bool result = validator.Validate("MySuperSecretPassword");

First off, a settings class is created, then various options are set. If you don’t set any options then the validator allows any password.

In this example the settings mandate the a password must be at least 6 characters, it must have a letter and it must have a number, and it must not appear in the built in list of the most common 500 passwords.

Then the Validator is created and passed the settings that we’ve prepared.

Finally, the Validate method is called passing in the password that is to be validated. The result indicates whether the password passed or failed (in the example above, it failed as it does not contain a number).

Settings from the config file

If you prefer to have the settings for the validator in the config file then you can instantiate a Validator without passing anything to its constructor and it will use the settings in the config file instead.

It should go without saying that you should only put the settings in the config file in a secure environment.

To use settings in the config file you must set up a the section where the settings will go, and then create the section with the settings in it.

To define the section:

<configSections>
<!-- Set up other config sections here—>
   <sectionGroup name="passwordValidation">
      <section name="rules" type="Xander.PasswordValidator.Config.PasswordValidationSection, Xander.PasswordValidator, Version=0.1.0.0, Culture=neutral, PublicKeyToken=fe72000dffcf195f" allowLocation="true" allowDefinition="Everywhere"/>
   </sectionGroup> </configSections>

An example of the config section itself:

<!-- The configuration section that describes the configuration for the password validation -->
<passwordValidation>
   <rules minimumPasswordLength="6" needsNumber="false" needsLetter="false" needsSymbol="false">
     <wordListProcessOptions checkForNumberSuffix="true" checkForDoubledUpWord="true" checkForReversedWord="true" />
     <standardWordLists>
       <add value="FemaleNames"/>
       <add value="MaleNames"/>
       <add value="MostCommon500Passwords"/>
       <add value="Surnames"/>
     </standardWordLists>
     <customWordLists>
       <add file="WordLists/MyCustomWordList.txt" />
       <add file="WordLists/MyOtherCustomWordList.txt" />
     </customWordLists>
   </rules> </passwordValidation>

The above example uses most options available out of the box that can be put in the config file. It is worth noting that some options are only available from the settings in the code, such as being able to specify custom classes that handle parts of the validation.

Have a play

If you want to try this out for yourself the two assemblies are available here. I will be putting this on NuGet soon.

Xander.PasswordValidator

I was looking for a way to validate passwords for an upcoming project because we need to do that better than we are, so I was looking for some .NET library to do that. Unfortunately, Google only turned up a ton of regular expressions which is all fine an well, but it wasn’t really what I was looking for. I really wanted to be able to check against a dictionary of potential passwords and have it reject a password because it was in the list and it also had to check against the regular rules of “must has a number”, and so on.

So, I created Xander.PasswordValidator which is available on GitHub.

The idea is that you can create a Validator object which can then be used to evaluate the suitability of potential passwords. When creating the validator you can pass in settings from code, or have it read settings from a config file. You can extend the functionality, so if there is something that you want to check that I’ve not thought about you have points where you can extend that. If the lists of forbidden passwords are not suitable then you can add your own lists, if the rules for checking against those lists are not suitable, then you can add your own ways to check.

Also, if you write web applications using ASP.NET MVC then there is a really simple attribute you can apply to a property in your model to have the validator check the value.

Over the next few days I’ll be writing more documentation for the project and showing what can be done with it in a series of blog posts. The “official” documentation will be on the project’s wiki at GitHub. An initial version of the built code will also be available in the next few days. However, you can get the code from GitHub and build it yourself if you so desire.

Quick guide to Geolocation in Javascript

In some modern browsers, such as Chrome and Firefox you can access the geolocation of the device. That is, where the device is physically located.

The main function for achieving this is getCurrentPosition, which doesn’t return a position as you might expect. Rather, it takes a callback (and optionally a second if you want to handle error conditions).

I’ve put together a small example page showing this, which I’ll now walk through.

In the example, when the user clicks on the button on the page it will attempt to get the physical location of the device. This may or may not work for several reasons. If it doesn’t work then the browser may not support it, or the user may refuse to give permission to the site, or the geolocation service may not be working.

This first bit of code checks to see if the browser supports the geolocation API and if it does calls the function to get the location passing in the callbacks for success and error handling.

if (navigator.geolocation) {
    navigator.geolocation.getCurrentPosition(successCallback, errorCallback);
} else {
    displayErrorMessage("The browser does not support the Geolocation API.");
}

In this small example, the successCallback simply fills various spans with the results in the position and creates a URL that links to google maps to display a pin at the coordinates.

function successCallback(position) {
    $("#latitude").html(position.coords.latitude);
    $("#longitude").html(position.coords.longitude);
    $("#accuracy").html(position.coords.accuracy);
    $("#displayMap").attr("href", "http://maps.google.com/?q=" + position.coords.latitude + "," + position.coords.longitude);
    $("#displayMap").removeClass("disabled");
}

The position has a timestamp and a set of coordinates. Since the geolocation may be cached the timestamp will give you an indication of how old the geolocation is.

The coords gives you various bits of information about the geolocation. The three values that will always be available are latitude, longitude and accuracy. The other values (such as altitude, heading and speed) may be nullable. The accuracy is in meters and can be used to gauge how good the lat/long is. The Lat/Long is in WGS84 decimal degrees.

In the event of an error, the errorCallback will receive some indication about what went wrong. The most common may be that the permission was denied, but other potential errors exist.

function errorCallback(error) {
    switch (error.code) {
        case error.PERMISSION_DENIED:
            displayErrorMessage("The request was denied. If a message seeking persmission was not displayed then check your browser settings.");
            break;
        case error.POSITION_UNAVAILABLE:
            displayErrorMessage("The position of the device could not be determined. For instance, one or more of the location providers used in the location acquisition process reported an internal error that caused the process to fail entirely.");
            break;
        case error.TIMEOUT:
            displayErrorMessage("The request to get user location timed out before the operation could complete.");
            break;
        case error.UNKNOWN_ERROR:
            displayErrorMessage("Something unexpected happened.");
            break;
    }
}

 

How your browser reacts to requests for geolocation

Your browser may give you some form of alert to indicate that the site is requesting the geolocation. Chrome, for example, displays a bar just under the omnibox

Chome asks if it is okay to use geolocation

If a site has permission to get the geolocation then the icon above will be displayed in the omnibox to the right of the URL. If not, the icon will have a red cross over it. You can click this icon to change the settings at any time.

 

Finally, if you want to read the spec in full, it is available here: http://www.w3.org/TR/geolocation-API/

2012 in review

The WordPress.com stats helper monkeys prepared a 2012 annual report for this blog.

WordPress emailed me a very nice summary of my blog stats for 2012. Actually, they emailed me twice because at the start of 2012 I self-hosted then I moved to WordPress when I realised that would be much cheaper and take a lot less effort. The only downside being that I had to give up Google Analytics. Below is a bit of a summary, I’ve tried to combine the numbers from both sides of the fence to give a more accurate picture:

 

4,329 films were submitted to the 2012 Cannes Film Festival. This blog had 103,000 views in 2012. If each view were a film, this blog would power 24 Film Festivals.In 2012, there were 36 new posts, growing the total archive of this blog to 492 posts.

These are the posts that got the most views in 2012.

  1. Custom error pages and error handling in ASP.NET MVC 3  May 2011
  2. Running Queries on Excel Spreadsheets using SQL Server 2008  August 2011
  3. Entity Framework: Unable to load the specified metadata resource.  July 2011
  4. Getting just the columns you want from Entity Framework, July 2011
  5. Kendo UI: Paging and accessing the filtered results in javaScript  July 2012

The top referring sites were:

  1. stackoverflow.com
  2. codeproject.com
  3. twitter.com
  4. social.msdn.microsoft.com
  5. blogs.teamb.com

Visitors came from 173 countries in all! Most visitors came from The United States. The United Kingdom & India were not far behind.