Developer Tools in IE Hides Bug

The other day I put some code on our UAT (User Acceptance Testing) server so that some new code could be tested and I started received a very alarming bug that one of the pages simply didn’t work at all. The page was in an administration section of the website and relied heavily on JavaScript even just to display the initial content (which was retrieved via an AJAX request)

As a quick initial smoke test I loaded the page up on my machine and it was working using my developer build. Then I looked at the page on the UAT server in case it was some quirk of the build and it was also working. I then noticed that the person doing the tests was running IE, so I though it might be a browser issue, so I loaded the page from the UAT server up again, this time using IE and it was still working on my machine.

Since it was lucky enough that the particular section was an admin section (which would be used only in-house) the people testing it were not actual customers of ours so I could walk over to their desk and ask for a demonstration in case there was some quirky step that had to be undertaken. When the page was loaded up I immediately saw that it didn’t work.

Being a developer my first instinct was to open up the Developer Tools in IE and have a look at what was happening. So I hit F12 and the reloaded the page…. and it started to work!  It worked with the Developer tools turned on!

I couldn’t fathom that out. What was so different about Internet Explorer when the Developer Tools were turned off so I started going through the various things that IE was showing me in the developer tools, stepping through code and watching the AJAX request go out then come back with data and start to process that data…. And then I noticed it. There was a line that said:

console.log("blah… blah…. blah…");

On a regular user’s machine the Developer Tools are never running, so it never has a console, so the JavaScript just broke.

Demonstrating the bug

How you you try this out? I’ve written a small demo to show what I mean. Obviously, you need to open it in Internet Explorer. I’ve tried it in IE 8.

If you have already opened the Developer Tools previously then you may find that they open automatically, which makes any testing impossible. Shutting them down and restarting IE doesn’t help. You have to go in to the registry and manually disable the developer tools.

To disable the Developer Tools in Internet Explorer you need to edit the system registry. Open up regedit and navigate to HKCU\Software\Microsoft\Internet Explorer\IEDevTools then create a DWORD called Disabled and give it the value of 1.

Once you have disabled the Developer Tools you’ll see that the page displays the text:

This paragraph has been update by javascript.

This paragraph is not yet updated by javascript, and if the Developer Tools are not present, it won’t update.

And a small warning triangle appears in the status bar of IE. Double clicking on the warning triangle brings up a dialog with some error information that looks like this:

To re-enable the Developer Tools simply delete that setting and restart Internet Explorer, press F12 to bring up the developer tools then open the demonstration page again.

Now the text displayed on the page reads:

This paragraph has been update by javascript.

This paragraph has also been updated by javascript, indicating the developer tools are present

Preventing this bug

Obviously running functions on the console object is not all that desirable in a production system so the idea is to remove all those calls so that IE won’t crash. If you find that is impractical you could put in some JavaScript before other JavaScript is run such as the following:

if (window.console === undefined) {
  console = {};
  console.log = function(){};
}

This will ensure that if a console object does not exist then one is created and a dummy function is attached to it. If you use other functions on the console object then you should add them also in a similar way.

The above demonstration has been updated to show this console “protection” in action.

 

RavenDB on IIS: Cannot access file, the file is locked or in use

I came across another issue with trying to get RavenDB working through IIS. When the process started up I got the error message “Cannot access file, the file is locked or in use”.

The stack trace looked like this:

[EsentFileAccessDeniedException: Cannot access file, the file is locked or in use]
   Microsoft.Isam.Esent.Interop.Api.Check(Int32 err) in C:\Work\ravendb\SharedLibs\Sources\managedesent-61618\EsentInterop\Api.cs:2736
   Raven.Storage.Esent.TransactionalStorage.Initialize(IUuidGenerator uuidGenerator) in c:\Builds\RavenDB-Stable\Raven.Storage.Esent\TransactionalStorage.cs:205

[InvalidOperationException: Could not open transactional storage: C:\inetpub\ravendb\Data\Data]
   Raven.Storage.Esent.TransactionalStorage.Initialize(IUuidGenerator uuidGenerator) in c:\Builds\RavenDB-Stable\Raven.Storage.Esent\TransactionalStorage.cs:220
   Raven.Database.DocumentDatabase..ctor(InMemoryRavenConfiguration configuration) in c:\Builds\RavenDB-Stable\Raven.Database\DocumentDatabase.cs:185
   Raven.Web.ForwardToRavenRespondersFactory.Init() in c:\Builds\RavenDB-Stable\Raven.Web\ForwardToRavenRespondersFactory.cs:84
   Raven.Web.RavenDbStartupAndShutdownModule.b__0(Object sender, EventArgs args) in c:\Builds\RavenDB-Stable\Raven.Web\BootStrapper.cs:13
   System.Web.SyncEventExecutionStep.System.Web.HttpApplication.IExecutionStep.Execute() +80
   System.Web.HttpApplication.ExecuteStep(IExecutionStep step, Boolean& completedSynchronously) +270M

It turns out that I hadn’t given the account IIS was using access to the directory… Or rather, I had given access, I just didn’t tell IIS about the account so it was using the default account.

To fix the issue:

  • Go into the Application Pools section of IIS.
  • Click the Application pool used by the RavenDB site

  • Press “Advanced Settings…” on the right side of the dialog.
  • Ensure the correct account is set up for the “Identity” in the “Process Model” section.

  • Once you’ve set up the correct user, press "OK" for each of the dialogs and everything should be ready.

Getting RavenDB working on IIS – 500.19

While trying to get RavenDB working in IIS I ran into a problem. I got the following error from IIS

HTTP Error 500.19 – Internal Server Error

The requested page cannot be accessed because the related configuration data for the page is invalid.

Module IIS Web Core
Notification BeginRequest
Handler Not yet determined
Error Code 0x80070021
Config Error This configuration section cannot be used at this path. This happens when the section is locked at a parent level. Locking is either by default (overrideModeDefault="Deny"), or set explicitly by a location tag with overrideMode="Deny" or the legacy allowOverride="false".
Config File \\?\C:\inetpub\ravendb\web.config

Config Source

    6: 	<system.webServer>
    7: 		<handlers>
    8: 			<add name="All" path="*" verb="*" type="Raven.Web.ForwardToRavenRespondersFactory, Raven.Web"/>

I’ve ignored some of the less interesting parts of the error message.

This can be fixed in IIS itself.

  • Open up IIS, click on the top node of the tree on the left side (the one labelled with the machine name)
  • Then double-click on the item in the centre pane marked “Feature Delegation”

  • Then find the entry marked “Handler Mappings” and set the delegation to “Read/Write” using the action links on the right side of the dialog.

When this is done the error moves on to the next part of the web.config

Config Source

    9: 		</handlers>
   10: 		<modules runAllManagedModulesForAllRequests="true">
   11: 			<remove name="WebDAVModule" />

At this point you have to also set the “Modules” to “Read/Write” also

Once that was set, I was good to go.

Git cheat sheet

Here is my little cheat sheet for working with git at the command line.

Check the status of the current repository:

git status

This will return details of which files are modified and any files that are untracked, etc.

Add a file to the repository:

git add <filename>

You can also replace filename with a dot (.) to include all untracked files. Use git status before to find out what files are currently untracked.

Commit the changes to the local repository:

git commit -m "<commit message>"

This will only commit the changes as far as the local repository.

Send the changes back to the server:

git push origin master

origin is the name of the remote location of the repository. It is more-or-less, by convention, where you cloned your local repository from.

master is the name of the branch. By convention this is your default branch.

Revert a file to the version at the previous commit:

git checkout -f <filename>

Delete files from the repository

git rm <filename>

If you want to delete an entire directory you need to add the -r (recursive) flag.

git rm -r <folder-path>

You may also need to choose between keeping the files on the disk and removing them altogether. In which case you want either --cached, to remove them from the repository but keep them on disk, or -f to force the removal of the file from disk.

Umbraco installation woes

Recently, I created an Umbraco site on one machine and I wanted to move it to another. I say site, there was nothing in it. It was really a basic database, but I’d configured it with just an empty install before I zipped up the solution and all the files in it.

When I unzipped the files on to a new machine every time I tried to run the Umbraco install routine it would redirect me to a log on page even although the site had only a completely empty database to connect to (not a single table because the installation hadn’t run yet).

The reason is that on installation part of the web.config is updated.

<add key="umbracoConfigurationStatus" value="4.8.0" />

And that indicates to Umbraco that the installation is complete. So, allow installation to proceed normally remove the value and the installation can start normally, like this:

<add key="umbracoConfigurationStatus" value="" />

Tip of the day: When other things populate elements on web pages

I work in a web team of 6 developers and we have a diverse range of skills. Some are more skilled dealing with the front end, the UI, and working with HTML, JavaScript and CSS. Others are better with the back end of things, the business logic and dealing with databases. However, there is an overlap.

One thing I noticed earlier this year when we took on our first front end specialist is that she did all sorts of things to optimise the HTML, CSS and JavaScript. It was wonderful, the application worked a lot better and looked a lot nicer. Unfortunately, some of the optimisations broke things because we’d been populating divs and spans from Ajax calls, third party code, or other bits of JavaScript. Because we knew where the data was coming from, having the full view as we did, we’d not bother commenting the HTML (and HTML comments waste bandwidth, right?)

It became obvious that something was going to need to be done if we were going to work with a specialist like this. It occurred to us that the ASP.NET rendering engine allows us to create server side comments that are not transmitted to the client. That helped keep the HTML clean on the client and allowed us to indicate to our specialist that there was a div or span that is needed (and shouldn’t be optimised away)

<div id="sizeSurchargesInfo"><%-- Populated by an AJAX Callback --%></div>

In hindsight, it seems rather obvious to simply comment the code. However, in these enlightened times it does seem unnecessarily frowned upon.

Getting umbraco up and running in with MVC 4

In this post, I’ll look at getting Umbraco and MVC to play nice with each other in the same project.

Installing Umbraco 4.8

First off create a Web Application project in Visual Studio. For this example, I’m just going to create the project as “UmbMvc”.

Visual Studio 2010: New Project
Visual Studio 2010: New Project

Once Visual Studio has created the project, delete most of its content. We’re doing this because we don’t have fully empty projects. If you were doing this in VS 2012 you could have selected the Empty Web Application project instead.

The Solution explorer should look like this when your done:

Visual Studio 2010: Solution Explorer
Visual Studio 2010: Solution Explorer

Next up, Umbraco has to be installed. This can be done with NuGet. I used the Package Manager Console, which can be accessed from the Tools menu:

Visual Studio: NuGut Package Manager Console
Visual Studio: NuGut Package Manager Console

Then typed Install-Package UmbracoCms to install the package and its dependencies. The output looks like this:

PM> Install-Package UmbracoCms
'UmbracoCms.Core (= 4.8.0)' not installed. Attempting to retrieve dependency from source...
Done.
Successfully installed 'UmbracoCms.Core 4.8.0'.
Successfully installed 'UmbracoCms 4.8.0'.
Successfully added 'UmbracoCms.Core 4.8.0' to UmbMvc.
'web.config' already exists. Skipping...
Successfully added 'UmbracoCms 4.8.0' to UmbMvc.

Don’t worry about the message about web.config. It will write the necessary detail into the web.config file for you.

If you prefer to use the NuGet dialog, you can search for “UmbracoCms” and install the package from there. It will download and install the dependencies for you there too.

NuGet Package Manager Dialog
NuGet Package Manager Dialog

At this point you can run up Umbraco to configure it and set the databases up and so on. When you’ve finished this process you’ll arrive at the Umbraco administration area. At this point you want to stop the app from running in Visual Studio.

Wiring up MVC 4

Next up is to get MVC installed. For this I’m taking the advice on Aaron Powell’s blog, so go visit there for the detail. (Start at the section marked “Getting MVC installed”). I’ve added my own notes below for some differences I found between our experiences.

Installing ASP.NET at the time of writing installs MVC 4:

PM> install-package microsoft.aspnet.mvc
'Microsoft.AspNet.WebPages (= 2.0.20505.0)' not installed. Attempting to retrieve dependency from source...
Done.
'Microsoft.Web.Infrastructure (= 1.0.0.0)' not installed. Attempting to retrieve dependency from source...
Done.
'Microsoft.AspNet.Razor (= 2.0.20505.0)' not installed. Attempting to retrieve dependency from source...
Done.
Successfully installed 'Microsoft.Web.Infrastructure 1.0.0.0'.
Successfully installed 'Microsoft.AspNet.Razor 2.0.20505.0'.
Successfully installed 'Microsoft.AspNet.WebPages 2.0.20505.0'.
Successfully installed 'Microsoft.AspNet.Mvc 4.0.20505.0'.
Successfully added 'Microsoft.Web.Infrastructure 1.0.0.0' to UmbMvc.
Successfully added 'Microsoft.AspNet.Razor 2.0.20505.0' to UmbMvc.
Successfully added 'Microsoft.AspNet.WebPages 2.0.20505.0' to UmbMvc.
Successfully added 'Microsoft.AspNet.Mvc 4.0.20505.0' to UmbMvc.

Because of the way paths work in Umbraco, I was either going to have to reserve every controller and area name in the umbracoReservedPaths configuration element, or create a prefix. I decided it was probably best to create a prefix, that way I only have to modify the config once and everything else simply works after that. So, my RouteSetup class looked like this:

using System.Web;
using System.Web.Mvc;
using System.Web.Routing;

[assembly: PreApplicationStartMethod(typeof(UmbMvc.App_Start.RouteSetup), "Setup")]
namespace UmbMvc.App_Start
{
  public class RouteSetup
  {
     public static void Setup()
     {
       RouteTable.Routes.MapRoute(
         "Default", // Route name
         "x/{controller}/{action}/{id}", // URL with parameters
         new {controller = "Home", action = "Index", id = UrlParameter.Optional} // Parameter defaults
         );
     }
  }
}

Note the “x” at the start of the URL part of the route.

My umbracoReservedPaths config element now looks like this:

<!-- Remember to add into the umbracoReservedPaths every route that MVC wants to take. 
     It may be better to create a prefix so you only have to do this the once.-->
<add key="umbracoReservedPaths" value="~/umbraco,~/install/,~/x" />

Fixing up the web.config file

From Aaron’s blog post, I still couldn’t quite get it to work. I got an error message that read: Compiler Error Message: CS0234: The type or namespace name ‘Helpers’ does not exist in the namespace ‘System.Web’ (are you missing an assembly reference?)

I found that I needed to add the following to the <assemblies> section of the web.config file:

<add assembly="System.Web.WebPages, Version=2.0.0.0, Culture=neutral, PublicKeyToken=31BF3856AD364E35" />
<add assembly="System.Web.Helpers, Version=1.0.0.0, Culture=neutral, PublicKeyToken=31BF3856AD364E35" />

That helped, but I was still getting error an error message. This time it was Compiler Error Message: CS0234: The type or namespace name ‘Ajax’ does not exist in the namespace ‘System.Web.Mvc’ (are you missing an assembly reference?)

I found that I also needed to set the reference to System.Web.Mvc to “Copy Local”

Visual Studio 2010: System.Web.Mvc Copy Local
Visual Studio 2010: System.Web.Mvc Copy Local

Then when I ran the application and went to the URL http://localhost:60445/x/Home, I got a page back that said: Hello I’m a razor view.

This is finally what I expected.

Fixing up the project type.

One last thing, Aaron also mentions that you’ll get various errors in the views in Visual Studio because the project type was a Web Application not an MVC Web Appliction project. Although it doesn’t stop the application from running correctly, it is very disconcerting to see. He doesn’t give a solution to that. To solve this, you need to add a GUID in the csproj file.

To do this, you need to right click on the web project and click “Unload Project”, when it has unloaded, right-click again and click “Edit xxx.csproj”.

Look for the element named “ProjectTypeGuids” and add in the guid: {E3E379DF-F4C6-4180-9B81-6769533ABE47}. The whole line should now read:

<ProjectTypeGuids>{E3E379DF-F4C6-4180-9B81-6769533ABE47};{349c5851-65df-11da-9384-00065b846f21};{fae04ec0-301f-11d3-bf4b-00c04f79efbc}</ProjectTypeGuids>

Save the file, then right-click the project in the solution and click “Reload project”. It will prompt you to close the text version of the project file, and then the project will be loaded back. Now you should not have any issues with the views finding false errors, such as not being able to resolve ViewBag.

Optimising clustered indexes in SQL Server 2008

I’ve just found a script on another blog to go through all the clustered indexes in a SQL Server database and rebuild them in order to reduce fragmentation and improve the disk IO needed to get the data. The original script didn’t take into account tables in different schemas so I updated it. I also added some metrics to it so I could get a sense of how long the operation takes on each table.

The script can take quite a while to run. On my database it took over 6 minutes just to initially run the query to work out what needed rebuilding, and each index can take several seconds (or possibly more if you have a lot of data) on its own.

The new script is here:

SET NOCOUNT ON
DECLARE @Schema SYSNAME;
DECLARE @Table SYSNAME;
DECLARE @Index SYSNAME;
DECLARE @Rebuild NVARCHAR(4000)
DECLARE @StartTime DATETIME = GETUTCDATE();

PRINT (CONVERT(NVARCHAR(100), GETUTCDATE(), 113) + ' : Rebuild all indexes with over 10% fragmentation.')

DECLARE DB CURSOR FOR 
SELECT SS.name [schema], SO.Name [table], SI.Name [index]
FROM SYS.dm_db_index_physical_stats (DB_ID(), NULL, NULL , NULL, 'LIMITED')
INNER JOIN SYS.objects SO ON SO.object_id=SYS.dm_db_index_physical_stats.object_id
INNER JOIN SYS.schemas SS ON SO.schema_id=SS.schema_id
INNER JOIN SYS.indexes SI ON SI.index_id=SYS.dm_db_index_physical_stats.index_id AND 
SI.object_id=sys.dm_db_index_physical_stats.object_id 
-- Begin select only clustered indexes Index_id = 1
AND SI.Index_id = 1
-- End select only clustered indexes Index_id = 1
WHERE avg_fragmentation_in_percent > 10.0 AND SYS.dm_db_index_physical_stats.index_id > 0
ORDER BY SO.Name 

OPEN DB
FETCH NEXT FROM DB INTO @Schema, @Table, @Index
WHILE @@FETCH_STATUS = 0
BEGIN
  SET @Rebuild = 'ALTER INDEX ' + @Index + ' ON ' + @Schema + '.' + @Table + ' REBUILD'

  PRINT (CONVERT(NVARCHAR(100), GETUTCDATE(), 113) + ' : ' + @Rebuild)

  -- Comment out the following line to see what tables would be affected without rebuilding the indexes
  EXEC SP_EXECUTESQL @Rebuild

  FETCH NEXT FROM DB INTO @Schema, @Table, @Index
END
CLOSE DB
DEALLOCATE DB

DECLARE @Duration DATETIME = GETUTCDATE() - @StartTime;
PRINT (CONVERT(NVARCHAR(100), GETUTCDATE(), 113) + ' : Finished. Duration = '+CONVERT(NVARCHAR(100), @Duration, 114))

SET NOCOUNT OFF

HTML5 text boxs for numbers and email addresses

In HTML5 there are a few new types of text box available that can help with mobile development. For example, on mobile devices it can be useful to have the on-screen keyboard be defaulted to display a specific configuration when it is displayed, such as the numeric keypad.

Not all browsers support these new text box types at the moment, but it seems that if the browser doesn’t understand the type assigned to an input element it renders it as a textbox anyway.

Regular Textbox

By regular text box I mean something rendered with something like the following HTML:

<input type="text />

Normally, I use SwiftKey, but for this demonstration I’ve switched to the default Android keyboard.

Keyboard for regular text box

Numeric Textbox

You can set the default keypad to numeric by specifying an input like this:

<input type="number" />

Numeric input displaying numeric keypad

Email text box

Finally, there is an email input type that configures the keyboard to display a key for “.com” and a specific key for the “@” as well as dedicated keys for the “-” and “_”  next to the space bar (at least that’s how the default Android keyboard configures itself in this mode – YMMV). The input element looks like this:

<input type="email" />

Email keyboard

Finally

If you want to try this out for yourself, there is an example available.

Kendo UI: parse – preprocessing data

When retrieving data, it may not be formatted as you would need it. Most obviously, dates are the most likely candidates as the grid can work with them much more easily if they are javaScript Date objects rather than any text or numeric representation. It should be noted however, that if by simply telling the dataSource configuration that the schema of the a specific field is a date then it may be able to work out the format for itself and you don’t need a parse function to help. However, for this example, assume you must convert the type of the value.

dataSource : schema : parse

The dataSource configuration allows you to specify a function that is called when the data needs to be preprocessed in some way.

The parse function takes a parameter where by it passes the object containing the data. The function must return the processed data. In my example I’ve simply replaced the values in the existing structure with the processed version.

function preprocessData(data) {
  // iterate over all the data elements replacing the Date with a version
  // that Kendo can work with.
    $.each(data, function(index, item){
      item.Date = kendo.parseDate(item.Date, "yyyy-MM-dd");
    });
    return data;
}

The JSON structure contains a date in a string with a specific format containing a 4 digit year, followed by a two digit month, followed by a two digit day, separated by dashes. However, the grid can work with dates more easily if they are Date objects, which is what the kendo.parseDate() function returns.

Dealing with percentages

In a previous post I mentioned that you can format a number as a percentage by using a specific format in the kendo.toString() function call. Unfortunately, that may not be the best solution in all cases. If your data is not going to be filtered and it is in range of 0 to 1 representing 0% to 100% then that solution is fine. However, if you want to filter on the data then you probably don’t want to do that, as you’d have to enter set up the filter in the same way as the source data – and it is not intuative for the user to have to type “”0.5” when they need “50%”.

What you can do instead is ensure that the data is in the form that 100.0 is 100%, and so forth. You can use the parse function to coerce the data if you need to do that. Once you have this the filters become more intuative from the user’s perspective. Also, instead of using the built in format for parsing percentages you will need to use your own, such as “0.0”, which ensures that the value has one digit after decimal point. For example:

template:"#= kendo.toString(Rpi, \"0.0\") #%"

Filtering on a percentage column

The grid configuration

$(function(){
  var data = getData(); // From the economic-data.js file
  $('#MyGrid').kendoGrid({
    dataSource: {
      data: data,
      pageSize: 10,
      schema: {
           parse: function(data){
             return preprocessData(data);
           },
          model: {
          fields: {
            Date: {type: "date" },
            Rpi: {type: "number" },
            Cpi: {type: "number" },
            BoeRate: {type: "number" }
          }
        }
      }
    },
    filterable: true,
    columnMenu: false,
    sortable: true,
    pageable: true,
    scrollable: false,
    columns: [ 
      { field: "Date", template: "#= kendo.toString(Date, \"MMM yyyy\") #" }, 
      { field: "Rpi", title: "Inflation (RPI)", template:"#= kendo.toString(Rpi, \"0.0\") #%" }, 
      { field: "Cpi", title: "Inflation (CPI)", template:"#= (Cpi !== null ? kendo.toString(Cpi, \"0.0\")+\"%\" : \"-\") #" },
      { field: "BoeRate", title: "Base Rate", template:"#= kendo.toString(BoeRate, \"0.0\") #%" }
    ]
  });
});

More information

For this post the data is a mash up of UK Inflation data since 1948 and Bank of England Base Rates since 1694. I’ve only used the intersecting dates of both datasets.

The economic-data.js file is available as a github gist.

There is also a working example of this code.