Archive for the 'Technology' Category

AUP Informant 2 - Client Development

Published
by
Paul
on August 19, 2008
in Technology
. 0 Comments

I admit that when it comes to design I’m a rabid perfectionist. The moment I’ve finished working on something I’m already working on something better and something for after that.

For the past couple of months I’ve turned my attention to the AUP Informant client interface. Let’s have a look at the current version before I show you what’s new.

Give it a click to expand the image. What’s wrong with it? At first glance, not too much. There are only 3 major elements after all - a graphic, a web browser control and a button group. Something doesn’t seem right though and here’s my list of gripes:

  • Small size means a cramped interface
  • Duplication of text in the titlebar and graphic
  • Horizontal scrollbar not needed
  • Unrequested dialogue box can be confusing

Let’s focus on that last one, since it’s the most important. I thought through the experience of a user logging on for the first time and realised that a strange dialogue appearing just as I’m getting started would throw me. While having the dialogue appear is unavoidable I’ve worked hard to make it friendlier, easier to understand and quicker to get rid of.

I drew up a few goals for the new client design

  • Must integrate well into Vista as well as XP
  • Must look similar to an OS dialogue
  • Should use reassuring, simple language
  • Keep the number of options to a minimum
  • Since the dialogue is unmovable, emphasise it’s neccessity with a background

Here’s what I’ve come up with. I hope you like it!

Tying it all together

Published
by
Paul
on July 5, 2008
in Technology
. 0 Comments

All applications fit neatly into one of two categories - those that repackage easily and those that are close to impossible. For the past 2 months I’ve been dealing with one of the latter, ArcGIS by Esri, a geographical information system. Basically, it’s a suite of apps for working with mapping data that’s split into many components.

The problems with packaging aren’t the fault of Esri at all. In fact, unlike a lot of companies, an MSI is provided on the disk which would normally make life easier. On an RM network however, the procedure is a little different from a vanilla deployment. Instead of using active directory to deploy the MSI directly, you have two options:

  1. Set up a “package build” workstation  - essentially a computer with a minimal number of packages. You then use the RM Application Wizard to perform a “before scan” of the station, install your software and then run an “after scan”, capturing the changes and creating an MSI file. On the plus side, this lets you change program settings before deployment which is a nice touch. One of the problems I often run into is that occasionally changes are missed, which makes some software think the install is corrupt. The other common problem is that user settings are often pulled into the MSI and during deployment these are sometimes unable to be set and the install will fail.
  2. The Application Wizard is also able to accept MSIs directly, repackaging them on the fly and generating shortcuts and other required files in the RM package format.

Unfortunately, both of these methods failed for ArcGIS. I suspect the software is just too big and complex for a simple repackage job - a basic install is over 1.4GB, with files and registry keys all over. That’s 1.4GB of software - no maps, no data! The MSI deployment would always fail with an MSIInstaller error 11606, complaining that it was unable to find “WindowsVolume%\”.

I decided the best way forward would be to somehow install the original MSI unedited. The RM system does support this by way of its package.ini files. Using these, you can set EXEs to run and MSIs to install. What I couldn’t do was pass the /qb switch to the MSI in order to do a typical, silent install. It was time for some scripting…

The workaround was to tell the package manager to execute my custom batch file, which itself called MSIInstaller /I setup.msi /qb after verifying the sizes of certain installer files. Frustratingly, when the batch file finishes, it seems to return a code that the RM package manager interprets as a failed install! Luckily, though there’s a switch in the package.ini file to ignore return codes and this seemed to do the trick. Big thanks to whoever at RM implemented that feature!

I also had to write an uninstaller batch file, again calling MSIInstaller with the correct product code, since the RM system now has no record of the particular MSI that has been installed.

In an ideal world, that would be it but I also had to install a service pack that was supplied as an MSP file. I couldn’t use method 1 above to package it because I couldn’t do the original install that way and method 2 doesn’t support MSP files. Neither, it seems, do the RM package.ini files, as you can only specify an MSI to run, not an MSP. Batch file time again, this time executing the MSP in the same method as above. Finally, a working installer that brings together the two worlds of vanilla MSIs and the RM system.

It took a little longer than our usual 2 day turnaround on software packaging but I’m pleased with the result, from now on it’s quite simple to install generic MSI files with just a little batch editing. Don’t get me wrong, the RM system has its advantages but this method is sometimes much easier than trying to work around strange repackaging errors.

EduSweep Beta 6 Released

Published
by
Paul
on May 11, 2008
in Technology
. 1 Comment

After around 3 weeks of refining code, I’m happy to announce the final beta release of EduSweep. This is the first build that is of proper release quality and from now, only major bugs will be fixed before 1.0.0.

The following bugs have been fixed in this release:

  1. Large sets of results (> 5000 results) cause the results viewer to take a few seconds to appear
  2. Scan engine may fail to recover after multiple inaccessible folders are encountered
  3. Results viewer controls behave properly when selecting multiple items and only some have been deleted or quarantined
  4. A proxy authentication error is encountered while updating the definitions
  5. Two minor bugs that were not fatal but could slow down the scan process

    There are also some notable improvements, some of which I described in detail in the previous post:

    1. The responsiveness of the results viewer has been improved while loading information for files
    2. Online update has been overhauled with a scan log and better handling of proxies
    3. Log files are now written in HTML format and give more detail about scan errors

      Go ahead and grab the download! :)

      EduSweep Beta 6 Changes

      Published
      by
      Paul
      on May 7, 2008
      in Technology
      . 0 Comments

      Having taken the overwhelmingly positive feedback from Beta 5 testing over at EduGeek.net, I’ve been hard at work on the next and final beta build. Let’s have a look at what’s changed in this version as it nears release:

      New logging features in the online update module

      More robust definition updates

      The online update feature has had a major re-write, bringing a realtime log on-screen and an error log to help trace any issues. The proxy handling code has been updated and should fix the issue some people were seeing with their Squid proxies and NTLM authentication.

      On startup, the installed definitions are now verified so that if any definitions are missing or damaged, they will be redownloaded as part of the update. The definition files are now also tested before a new scan begins.

      Log files are much, much better

      Previously, log files from a scan would be saved with a rather cryptic filename as text files in the ‘logs’ folder. The actual content of a log looked something like this:

      #### Scan Summary ####

      Scan Date: 10/04/2008 12:02:07
      Files Scanned: 518
      Errors: 0
      Detected Items: 4

      #### Detected Items ####

      \\nbl-sr-02\student$\My Settings\Favorites\proxy\Freedom Proxy.url
      \\nbl-sr-02\student$\My Settings\Favorites\proxy\The OpenDoorNetwork -
      Unlock The Internet With Us! Proxy Sites For Work and School - Visit
      MySpace, Bebo, Facebook and MORE!! F.url
      \\nbl-sr-02\student$\My Settings\Recent\HACKER.lnk
      \\nbl-sr-02\student$\My Settings\Recent\pc virus.lnk

      Scan completed in 2 seconds.

      I’ve been hard at work upgrading the logging code and it’s now able to write (100% valid) html log files, which are just so much easier to work with - now that you can load them in a browser it’s possible to print them with proper formatting and word wrapping, as well as perhaps placing them on a web server for easy access anywhere.

      There’s also some new information that’s written to the logs. First, the scan summary now includes the user that started the scan and the time taken. The scan summary looks like this now:

      Scan date: 06/05/2008 11:49:53

      Scan requested by: paulb

      Scan took 11 minutes and 14 seconds to complete.

      Number of files scanned: 144150

      Number of files detected: 166

      Second, and more importantly, the logs show detailed information on errors instead of just a count. Some of the more common errors you may encounter are:

      • The specified path, file name, or both are too long. The fully qualified file name must be less than 260 characters, and the directory name must be less than 248 characters.
      • Access to the path ‘C:\System Volume Information’ is denied.

      Both of these are normal on a standard filesystem. The second is often triggered by the scanner trying to access system folders and it will continue just fine after skipping them. If a folder is skipped that definitely isn’t a system folder, check that your user account has read access.

      The results viewer is much quicker

      Notice the amount of results and the progress bar for the thread

      It’s hard to design a viewer that has to handle several thousand files and stay responsive. Beta 5 could slow down quite a bit if your scan returned a lot of detected items. I’ve added some threads to make loading data and working with files feel much more responsive throughout.

      The progress bar bottom right shows the progress of gathering data for the files and populating the list.

      Framework folder fudges in .NET

      Published
      by
      Paul
      on May 5, 2008
      in Technology
      . 1 Comment

      I love the .NET Framework. I mean that purely in a platonic sense, of course - it’s not *that* good. :lol: As with all platforms though, it falls down in certain areas. Java has its slightly quirky date APIs and I think one of .NET’s quirks has to be folder handling.

      Back in my day…

      Like many people, for better or for worse, I started out programming in VB6 around 2003. I’d managed to write some fairly decent programs - decent more in functionality than code quality, I admit. When Visual Studio .NET 2003 came out, I started learning to program in C# on the then relatively young .NET framework. I was having so much fun, in fact, that I asked for Visual C# Standard for my birthday!

      Back then, the .NET framework made learning a new language much easier. There was a nice set of standard libraries, automated garbage collection and I found that a lot of the methods I was writing in VB were included as standard in the framework.

      Rose-tinted specs

      Recently, though, the framework has been getting on my nerves in a pretty big way. Sometimes the way it handles basic operations is odd, unreliable or just downright broken. One such problem is handling folders.

      EduSweep, my network sweeper, scans entire drives for unwanted files. The process it uses goes something like this:

      • Get the list of targets that the user selected
      • For each target, get all the subfolders and make a list of them
      • For each folder in the list, scan its files

      And that’s pretty much it; it’s not very complex, nor does it need to be. The framework has other ideas, however. By far the simplest way to get all the subfolders of a path would be to use GetDirectories like so:

      GetDirectories(“*.*”, System.IO.SearchOption.AllDirectories);

      That initially looks fine but let’s assume that we have a folder somewhere on the drive that is inaccessible due to insufficient permissions. A good example is the system folder “System Volume Information” in the root of drive C. The code attempts to get the subfolders for the folder but fails because the directory listing is denied.

      We can wrap that code in a try block, catching the exception, but then no results are returned at all! The method is essentially useless because you know that, as soon as it encounters a single folder it can’t access, it throws its toys out of the pram. The only alternative is to manually recurse through the directory tree, enumerating folders and subfolders along the way. :mad:

      Would it be so hard to skip folders along the way and return a string array of failed directories?

      EduSweep 2 Preview

      Published
      by
      Paul
      on April 20, 2008
      in Technology
      . 0 Comments

      EduSweep is just approaching release but I already have big plans for the next version. Here’s a look at just 3 of the many new features currently in development:

      Scan Profiles

      Let’s say your users are split over 5 servers and you scan them once a week. You shouldn’t have to babysit the process, adding the scan targets each time, just set up the scan how you like it - choose the definitions to use, the targets to scan and the action to take - and press save. The next time you want to scan those servers, all you need to do is select the profile and away you go!

      Integration with TRID

      TRID is an impressive project, looked after by Marco Pontello. It identifies a file’s type by matching patterns against a known list. For example, TRID detects MP3 files based upon the hex pattern 49 44 33 in the file header.

      For EduSweep, this unlocks a great deal of power. Your students have renamed all their WMVs to TXT files? No problem - the files will be detected and flagged by the TRID library in EduSweep. Truly intelligent file identification will be a great asset to the software and could well pick up on malicious files you never knew you had on your network!

      The TRID library is free for personal and educational use but I’ll be contacting Marco about integrating it into EduSweep, just to be on the safe side. If I do use it, I’ll be sure to send a healthy donation his way.

      Speed, speed and more speed!

      Some of the latest code I’m working on involves multi-threading. EduSweep 1.x is already threaded to keep the UI responsive but the scanner itself is just a single thread. An experimental piece of code that I’ve cooked up tests the scan targets to see if they live on different computers or different physical drives on the same computer. Physically separate targets will each get their own scan thread to really maximise the scan speed for multiple targets and large networks.

      But wait, there’s more!

      Those are just a handful of the cool features I’m working on. I’ll post an update soon with perhaps a few more nifty additions. Watch this space!

      iPlayer Out in the Open

      Published
      by
      Paul
      on April 2, 2008
      in Media and Technology
      . 0 Comments

      I take a look at what could happen over a couple of weeks if the BBC opened up their iPlayer platform tomorrow…

      BBC iPlayer Logo

      Despite taking some heavy flak from free software proponents and non-Windows users, the BBC’s iPlayer has turned out to be quite a success, thanks in part to a recent deluge of advertisements. Convenience, it seems, has trumped any DRM worries for the time being. Where users of non-Windows platforms are left out in the cold is access to the full-resolution downloads, which are currently Windows-only. Users of Macs or Linux machines are limited to the lower quality, streaming Flash player. Streaming programmes also expire long before the downloaded versions.

      As we’ve seen during the past few weeks, the demand for unrestricted, platform agnostic access is alive and well, as evidenced by the various hacks which have surfaced. The BBC has been playing cat and mouse for a while now with software developer Paul Battley, author of the iPhone spoofing hack, which enables users on any computer to download DRM-free, H264 encoded files designed for the iPhone. Also available for users who haven’t updated to Windows Media Player 11 is the FairUse4WM tool, which is able to strip the DRM from most WMV files, leaving, in effect, full quality files with no expiration date. These can then be passed freely between different users, computers and devices.

      So why does the BBC insist on keeping the locks firmly on its content? The official line is due to licensing issues, particularly with older content, which is understandable. Often, it’s hard enough to agree on the licensing terms for broadcast, let alone for distribution in the brave new world of online media. A second issue is that of content expiry. Currently, most iPlayer programmes expire after 30 days. With DRM-free media this would certainly be impossible to enforce. It already is. Current DRM can be circumvented easily, and the programmes available on iPlayer can also be freely recorded on PCs, VCRs and DVD recorders, all of which produce non-expiring copies. What I’d like to see is an agreement between content producers and the BBC which completely opens up the iPlayer platform. For the moment, that’s something of a Utopian vision but given the ubiquity of BBC content on P2P networks, all that remains is for the BBC and its content partners to capitalise on the true potential of the web as a delivery mechanism for TV.

      “When information is generally useful, redistributing it makes humanity wealthier no matter who is distributing and no matter who is receiving.” - Richard Stallman

      So, what would happen if the BBC opened up iPlayer completely? What would the response be over the first couple of weeks? My vision of it is as follows:

      Week One

      On the first day, the BBC updates its website with programme download links in multiple formats, discarding the crippled Windows Media files. The files on offer are:

      • For Windows: Windows Media Video (of the DRM-free variety)
      • For Mac: H264 encoded Quicktime
      • For Linux: OGG Theora and H264

      The BBC already delivers its WMV content at a bitrate of around 1Mbps. Bearing in mind that this is standard definition content, the quality and bitrate are high. All the above formats offer great quality at similar bitrates. Allowing Mac and Linux-friendly formats gives users of those platforms access to video that is far higher quality than the streaming format provided by Flash.

      The existing Kontiki download manager is abandoned in favour of a cross-platform Java client. Users are still asked to seed downloaded content to reduce the BBC’s distribution costs, just as they are by the existing iPlayer client.

      The BBC sees an immediate benefit as Mac and Linux users flock to download and seed the full quality files. The streaming bandwidth required by the BBC is reduced as the percentage of viewers using the flash streaming service instead of the download service falls from its current level of around 70%.

      Week TwoBBC Four on a Portable Media Player

      iPlayer content begins to be shared more readily on popular BitTorrent sites such as Mininova. Users quickly begin to transcode and re-compress the original files to better suit a wide range of devices such as portable media players, PDAs and DVD players.

      Week three and beyond

      In order to further distribute its content, the BBC publishes a full specification and accompanying API for iPlayer, allowing third parties to develop interfaces for its content. The BBC’s radio streams are also added to the iPlayer platform. Linux developers are able to create iPlayer video and radio plugins for MythTV and Banshee. The BBC receives statistical information from its official BitTorrent network about the popularity of channels and content, allowing it to better fund new media.

      “[The BBC is] free from both political and commercial influence and answers only to its viewers and listeners.” - BBC Charter

      From there, the possibilities are even more exciting. The BBC is a unique media organisation and is perhaps more capable of redefining the playing field than commercial broadcasters. Certainly the BBC possesses amazing potential, let’s hope they unleash iPlayer’s.

      Looking at Wordpress 2.5

      Published
      by
      Paul
      on March 30, 2008
      in Personal and Technology
      . 0 Comments

      I’ve just moved up to Wordpress 2.5 and it’s very impressive. There’s been a lot of nice work done to the post editor and the theme is a lot nicer.

      The only problem I did encounter was that my wp-admin page was blank. Typing in another page, like wp-admin/plugins.php worked fine. gatewayy describes this issue at wordpress.org:

      What I mean by this is that when your view domain.com/wp-admin nothing is viewable except for a blank page but if you view a specific page within /wp-admin such as /wp-admin/plugins.php the page displays properly. I have tried deactivating all plugins but the issue still occurs. Are there any suggestions for possible troubleshooting for a resolution?

      Apparently this is because the dashboard page is now a widget page and some themes have incompatible widgets which can prevent it loading properly. If you’re having this problem, go to your /wp-admin/themes.php page and select the default. Apply it and all should be well again.

      If you’re considering upgrading, it’s very smooth on the whole. I deleted one or two directories and then just copied the whole package over the top. There’s a handy list of compatible themes here.

      Announcing EduSweep

      Published
      by
      Paul
      on March 21, 2008
      in Technology
      . 0 Comments

      EduSweep is a nifty piece of software for picking up nasties on your network, be it scripts, exes, media files or sensitive information. It’s based around a very fast scanning engine which you can easily customise. Even better, the whole thing will be openly released under the GPL 3!

      The content of this post has moved

      Please see the EduSweep project page.

      Standards Compliance in IE8

      Published
      by
      Paul
      on March 15, 2008
      in Technology
      . 1 Comment

      Of course I understand that this is an early beta release but it did make me laugh! I’m glad to see Microsoft working on standards compliance but, as the screenshot shows, they have a way to go yet.

      IE8 Beta 1