Archive

EduSweep Beta 6 Changes

Published
by
Paul
on May 7, 2008
in Technology
. 0 Comments

Having taken the overwhelmingly positive feedback from Beta 5 testing over at EduGeek.net, I’ve been hard at work on the next and final beta build. Let’s have a look at what’s changed in this version as it nears release:

New logging features in the online update module

More robust definition updates

The online update feature has had a major re-write, bringing a realtime log on-screen and an error log to help trace any issues. The proxy handling code has been updated and should fix the issue some people were seeing with their Squid proxies and NTLM authentication.

On startup, the installed definitions are now verified so that if any definitions are missing or damaged, they will be redownloaded as part of the update. The definition files are now also tested before a new scan begins.

Log files are much, much better

Previously, log files from a scan would be saved with a rather cryptic filename as text files in the ‘logs’ folder. The actual content of a log looked something like this:

#### Scan Summary ####

Scan Date: 10/04/2008 12:02:07
Files Scanned: 518
Errors: 0
Detected Items: 4

#### Detected Items ####

\\nbl-sr-02\student$\My Settings\Favorites\proxy\Freedom Proxy.url
\\nbl-sr-02\student$\My Settings\Favorites\proxy\The OpenDoorNetwork -
Unlock The Internet With Us! Proxy Sites For Work and School - Visit
MySpace, Bebo, Facebook and MORE!! F.url
\\nbl-sr-02\student$\My Settings\Recent\HACKER.lnk
\\nbl-sr-02\student$\My Settings\Recent\pc virus.lnk

Scan completed in 2 seconds.

I’ve been hard at work upgrading the logging code and it’s now able to write (100% valid) html log files, which are just so much easier to work with - now that you can load them in a browser it’s possible to print them with proper formatting and word wrapping, as well as perhaps placing them on a web server for easy access anywhere.

There’s also some new information that’s written to the logs. First, the scan summary now includes the user that started the scan and the time taken. The scan summary looks like this now:

Scan date: 06/05/2008 11:49:53

Scan requested by: paulb

Scan took 11 minutes and 14 seconds to complete.

Number of files scanned: 144150

Number of files detected: 166

Second, and more importantly, the logs show detailed information on errors instead of just a count. Some of the more common errors you may encounter are:

  • The specified path, file name, or both are too long. The fully qualified file name must be less than 260 characters, and the directory name must be less than 248 characters.
  • Access to the path ‘C:\System Volume Information’ is denied.

Both of these are normal on a standard filesystem. The second is often triggered by the scanner trying to access system folders and it will continue just fine after skipping them. If a folder is skipped that definitely isn’t a system folder, check that your user account has read access.

The results viewer is much quicker

Notice the amount of results and the progress bar for the thread

It’s hard to design a viewer that has to handle several thousand files and stay responsive. Beta 5 could slow down quite a bit if your scan returned a lot of detected items. I’ve added some threads to make loading data and working with files feel much more responsive throughout.

The progress bar bottom right shows the progress of gathering data for the files and populating the list.

Share this item:
  • Digg
  • del.icio.us
  • Reddit
  • Slashdot

Framework folder fudges in .NET

Published
by
Paul
on May 5, 2008
in Technology
. 1 Comment

I love the .NET Framework. I mean that purely in a platonic sense, of course - it’s not *that* good. :lol: As with all platforms though, it falls down in certain areas. Java has its slightly quirky date APIs and I think one of .NET’s quirks has to be folder handling.

Back in my day…

Like many people, for better or for worse, I started out programming in VB6 around 2003. I’d managed to write some fairly decent programs - decent more in functionality than code quality, I admit. When Visual Studio .NET 2003 came out, I started learning to program in C# on the then relatively young .NET framework. I was having so much fun, in fact, that I asked for Visual C# Standard for my birthday!

Back then, the .NET framework made learning a new language much easier. There was a nice set of standard libraries, automated garbage collection and I found that a lot of the methods I was writing in VB were included as standard in the framework.

Rose-tinted specs

Recently, though, the framework has been getting on my nerves in a pretty big way. Sometimes the way it handles basic operations is odd, unreliable or just downright broken. One such problem is handling folders.

EduSweep, my network sweeper, scans entire drives for unwanted files. The process it uses goes something like this:

  • Get the list of targets that the user selected
  • For each target, get all the subfolders and make a list of them
  • For each folder in the list, scan its files

And that’s pretty much it; it’s not very complex, nor does it need to be. The framework has other ideas, however. By far the simplest way to get all the subfolders of a path would be to use GetDirectories like so:

GetDirectories(“*.*”, System.IO.SearchOption.AllDirectories);

That initially looks fine but let’s assume that we have a folder somewhere on the drive that is inaccessible due to insufficient permissions. A good example is the system folder “System Volume Information” in the root of drive C. The code attempts to get the subfolders for the folder but fails because the directory listing is denied.

We can wrap that code in a try block, catching the exception, but then no results are returned at all! The method is essentially useless because you know that, as soon as it encounters a single folder it can’t access, it throws its toys out of the pram. The only alternative is to manually recurse through the directory tree, enumerating folders and subfolders along the way. :mad:

Would it be so hard to skip folders along the way and return a string array of failed directories?

Share this item:
  • Digg
  • del.icio.us
  • Reddit
  • Slashdot

EduSweep 2 Preview

Published
by
Paul
on April 20, 2008
in Technology
. 0 Comments

EduSweep is just approaching release but I already have big plans for the next version. Here’s a look at just 3 of the many new features currently in development:

Scan Profiles

Let’s say your users are split over 5 servers and you scan them once a week. You shouldn’t have to babysit the process, adding the scan targets each time, just set up the scan how you like it - choose the definitions to use, the targets to scan and the action to take - and press save. The next time you want to scan those servers, all you need to do is select the profile and away you go!

Integration with TRID

TRID is an impressive project, looked after by Marco Pontello. It identifies a file’s type by matching patterns against a known list. For example, TRID detects MP3 files based upon the hex pattern 49 44 33 in the file header.

For EduSweep, this unlocks a great deal of power. Your students have renamed all their WMVs to TXT files? No problem - the files will be detected and flagged by the TRID library in EduSweep. Truly intelligent file identification will be a great asset to the software and could well pick up on malicious files you never knew you had on your network!

The TRID library is free for personal and educational use but I’ll be contacting Marco about integrating it into EduSweep, just to be on the safe side. If I do use it, I’ll be sure to send a healthy donation his way.

Speed, speed and more speed!

Some of the latest code I’m working on involves multi-threading. EduSweep 1.x is already threaded to keep the UI responsive but the scanner itself is just a single thread. An experimental piece of code that I’ve cooked up tests the scan targets to see if they live on different computers or different physical drives on the same computer. Physically separate targets will each get their own scan thread to really maximise the scan speed for multiple targets and large networks.

But wait, there’s more!

Those are just a handful of the cool features I’m working on. I’ll post an update soon with perhaps a few more nifty additions. Watch this space!

Share this item:
  • Digg
  • del.icio.us
  • Reddit
  • Slashdot

iPlayer Out in the Open

Published
by
Paul
on April 2, 2008
in Media and Technology
. 0 Comments

I take a look at what could happen over a couple of weeks if the BBC opened up their iPlayer platform tomorrow…

BBC iPlayer Logo

Despite taking some heavy flak from free software proponents and non-Windows users, the BBC’s iPlayer has turned out to be quite a success, thanks in part to a recent deluge of advertisements. Convenience, it seems, has trumped any DRM worries for the time being. Where users of non-Windows platforms are left out in the cold is access to the full-resolution downloads, which are currently Windows-only. Users of Macs or Linux machines are limited to the lower quality, streaming Flash player. Streaming programmes also expire long before the downloaded versions.

As we’ve seen during the past few weeks, the demand for unrestricted, platform agnostic access is alive and well, as evidenced by the various hacks which have surfaced. The BBC has been playing cat and mouse for a while now with software developer Paul Battley, author of the iPhone spoofing hack, which enables users on any computer to download DRM-free, H264 encoded files designed for the iPhone. Also available for users who haven’t updated to Windows Media Player 11 is the FairUse4WM tool, which is able to strip the DRM from most WMV files, leaving, in effect, full quality files with no expiration date. These can then be passed freely between different users, computers and devices.

So why does the BBC insist on keeping the locks firmly on its content? The official line is due to licensing issues, particularly with older content, which is understandable. Often, it’s hard enough to agree on the licensing terms for broadcast, let alone for distribution in the brave new world of online media. A second issue is that of content expiry. Currently, most iPlayer programmes expire after 30 days. With DRM-free media this would certainly be impossible to enforce. It already is. Current DRM can be circumvented easily, and the programmes available on iPlayer can also be freely recorded on PCs, VCRs and DVD recorders, all of which produce non-expiring copies. What I’d like to see is an agreement between content producers and the BBC which completely opens up the iPlayer platform. For the moment, that’s something of a Utopian vision but given the ubiquity of BBC content on P2P networks, all that remains is for the BBC and its content partners to capitalise on the true potential of the web as a delivery mechanism for TV.

“When information is generally useful, redistributing it makes humanity wealthier no matter who is distributing and no matter who is receiving.” - Richard Stallman

So, what would happen if the BBC opened up iPlayer completely? What would the response be over the first couple of weeks? My vision of it is as follows:

Week One

On the first day, the BBC updates its website with programme download links in multiple formats, discarding the crippled Windows Media files. The files on offer are:

  • For Windows: Windows Media Video (of the DRM-free variety)
  • For Mac: H264 encoded Quicktime
  • For Linux: OGG Theora and H264

The BBC already delivers its WMV content at a bitrate of around 1Mbps. Bearing in mind that this is standard definition content, the quality and bitrate are high. All the above formats offer great quality at similar bitrates. Allowing Mac and Linux-friendly formats gives users of those platforms access to video that is far higher quality than the streaming format provided by Flash.

The existing Kontiki download manager is abandoned in favour of a cross-platform Java client. Users are still asked to seed downloaded content to reduce the BBC’s distribution costs, just as they are by the existing iPlayer client.

The BBC sees an immediate benefit as Mac and Linux users flock to download and seed the full quality files. The streaming bandwidth required by the BBC is reduced as the percentage of viewers using the flash streaming service instead of the download service falls from its current level of around 70%.

Week TwoBBC Four on a Portable Media Player

iPlayer content begins to be shared more readily on popular BitTorrent sites such as Mininova. Users quickly begin to transcode and re-compress the original files to better suit a wide range of devices such as portable media players, PDAs and DVD players.

Week three and beyond

In order to further distribute its content, the BBC publishes a full specification and accompanying API for iPlayer, allowing third parties to develop interfaces for its content. The BBC’s radio streams are also added to the iPlayer platform. Linux developers are able to create iPlayer video and radio plugins for MythTV and Banshee. The BBC receives statistical information from its official BitTorrent network about the popularity of channels and content, allowing it to better fund new media.

“[The BBC is] free from both political and commercial influence and answers only to its viewers and listeners.” - BBC Charter

From there, the possibilities are even more exciting. The BBC is a unique media organisation and is perhaps more capable of redefining the playing field than commercial broadcasters. Certainly the BBC possesses amazing potential, let’s hope they unleash iPlayer’s.

Share this item:
  • Digg
  • del.icio.us
  • Reddit
  • Slashdot

Looking at Wordpress 2.5

Published
by
Paul
on March 30, 2008
in Personal and Technology
. 0 Comments

I’ve just moved up to Wordpress 2.5 and it’s very impressive. There’s been a lot of nice work done to the post editor and the theme is a lot nicer.

The only problem I did encounter was that my wp-admin page was blank. Typing in another page, like wp-admin/plugins.php worked fine. gatewayy describes this issue at wordpress.org:

What I mean by this is that when your view domain.com/wp-admin nothing is viewable except for a blank page but if you view a specific page within /wp-admin such as /wp-admin/plugins.php the page displays properly. I have tried deactivating all plugins but the issue still occurs. Are there any suggestions for possible troubleshooting for a resolution?

Apparently this is because the dashboard page is now a widget page and some themes have incompatible widgets which can prevent it loading properly. If you’re having this problem, go to your /wp-admin/themes.php page and select the default. Apply it and all should be well again.

If you’re considering upgrading, it’s very smooth on the whole. I deleted one or two directories and then just copied the whole package over the top. There’s a handy list of compatible themes here.

Share this item:
  • Digg
  • del.icio.us
  • Reddit
  • Slashdot

Announcing EduSweep

Published
by
Paul
on March 21, 2008
in Technology
. 0 Comments

EduSweep is a nifty piece of software for picking up nasties on your network, be it scripts, exes, media files or sensitive information. It’s based around a very fast scanning engine which you can easily customise. Even better, the whole thing will be openly released under the GPL 3!

The content of this post has moved

Please see the EduSweep project page.

Share this item:
  • Digg
  • del.icio.us
  • Reddit
  • Slashdot

Standards Compliance in IE8

Published
by
Paul
on March 15, 2008
in Technology
. 1 Comment

Of course I understand that this is an early beta release but it did make me laugh! I’m glad to see Microsoft working on standards compliance but, as the screenshot shows, they have a way to go yet.

IE8 Beta 1

Share this item:
  • Digg
  • del.icio.us
  • Reddit
  • Slashdot

Released: AUP Informant

Published
by
Paul
on March 6, 2008
in Technology
. 8 Comments

AUP Informant is an application I cooked up over the past couple of weeks that you might find useful. It’s a program that runs at logon to display your Acceptable Usage Policy to students.

The content of this post has moved

Please see the AUP Informant project page.

Share this item:
  • Digg
  • del.icio.us
  • Reddit
  • Slashdot

I’m Going to Durham!

Published
by
Paul
on February 27, 2008
in Personal
. 1 Comment

It’s late so this post won’t be anywhere near eloquent but, put simply, I must’ve ticked some of the right boxes during the interview because this morning I got my acceptance letter from Durham university, much to my relief! It really is a lovely place, take a look at their video here: http://www.dur.ac.uk/tour/

What’s more, it’s fully catered so I won’t need to learn how to cook. Well, not just yet anyway. I’ll be leaving in October to go up there but my job ends in July so I ought really to plan a trip or two for those couple of months in between. Hmm…what to do…?

Share this item:
  • Digg
  • del.icio.us
  • Reddit
  • Slashdot