Harry Potter and the Half-Blood Prince

Seeing a book come to life on the silver screen can be a blessing or a curse. The blessing comes when we’re rewarded with a faithful adaptation that manages to reach beyond our own imagination; putting visuals on the screen that leave us in awe; telling the story we’re familiar with in a way that still manages to capture the drama and suspense of the written form. Peter Jackson managed it with The Lord of the Rings trilogy: we cry when Gandalf falls, even though, as readers of the book, we know it’s coming. The flight from the Balrog down the great steps and across the bridge of Khazad Dum is a perfectly executed scene that contains an immense buildup of drama and ends in a painful and heartfelt climax.

The curse is when we’re not rewarded with a faithful adaptation; when the source material isn’t taken seriously; when we’re presented with a story that is not as visually stunning as the one conjured in our mind; when important details are left out of the story and it leaves us feeling cheated and unfulfilled; when scenes that played out dramatically in written form are left to die on the screen, devoid of any real drama or emotion.

Harry Potter and the Half-Blood Prince is a strange mix of blessing and curse; it is a film that looks good and sounds good – it is polished in every way we’ve come to expect – and we think it should be good, but ultimately it falls short of greatness by a very long way. And it is only after we – the readers of the book – have had time to sit back and really digest the film do we realize just how much we’ve been cheated; just how must went missing.

Director David Yates and his special effects crew do a wonderful job of putting us squarely in the Harry Potter universe; we are greeted with all the familiar characters and locations and enough cameo performances to remind us just how big the tale is that we’re following through these six films. But he also manages to delete just about every important detail from this particular story, which leaves the whole thing feeling sterile and (to non-readers of the book) unnecessarily confusing.

I complained about this problem in my Order of the Phoenix review:

While the director David Yates and screenwriter Michael Goldenberg wisely trim the film of many of the side stories and sub plots from the book, it could have used another twenty or thirty minutes of development for the main story. Having read the book prior to seeing the movie, I noticed several scenes in the film that were exceptionally brief almost to the point of being unclear. Some viewers are going to feel like they’re missing out on a larger story (and they are).

Apparently, Yates didn’t learn his lesson with Order of the Phoenix. Harry Potter and the Half-Blood Prince is even more condensed than its predecessor; crucial information that pertains to the main story arc is simply not even presented to the viewer. And it’s a shame, really. Because Half-Blood Prince is easily one of the best Potter books and should have been one of the best films.

Readers of the book will recognize a vast array of missing and incomplete story elements: Harry’s infatuation with the Half-Blood Prince’s potions book that borders on hero-worship; how Harry comes to learn of the qualities of the Beazor that ultimately saves Ron’s life; Harry learning about the origins of the Inferi; Hermoine’s never ending attempts to decipher exactly what “Half-Blood Prince” means and discover the identity of the previous owner of the potions book; the seriousness of Dumbledor’s hand injury; how Dumbledor injured his hand; Dumbledor attempting to figure out how to remove the locket from the water basin… All important elements of the story in the book, and all missing from the film.

And yet, like its predecessor, Order of the Phoenix, so many of these missing or incomplete story elements could have been easily handled with some very simple and effective scenes. An additional 20 minutes of film could have done wonders. I wanted so badly, for instance, to listen as Dumbledor explained the concept of the Inferi to Harry, but that teaching moment never came. I waited patiently with excitement to see Dumbledor riddle-out the answer to the locket and the water basin, but instead the answer pops out of Dumbledor’s mouth about as brutally as it must have landed on the script page.

And then there is the real crime of the film: the ending. The moment of the film when it should have been built up and set upon the audience with the grandest of climaxes, it instead is treated with irreverence. The final crucial scenes had no impact and no weight.

As a grown man reading Harry Potter and the Half-Blood Prince, I wept when I set upon the dramatic ending. I was surprised, completely unprepared for what happened, and utterly saddened. I re-read the ending more than once because of the sheer weight of the drama; it was powerful and heartfelt and it hammered me emotionally. Everything about those final pages was perfect – the build-up, the execution, the final moments. It was all perfectly done, perfectly written by author J.K. Rowling. All David Yates had to do was put it on the screen.

And he didn’t even come close.

Harry Potter and the Half-Blood Prince should have been the best film of the series so far. It could have been – with a few more key moments sprinkled in and an ending that would have stayed faithful to the book, it would have been.

Unfortunately, that didn’t happen.

On a recent project I worked on I had a need to create a client-server application. What I was after was real-time updating of the clients when data on the server changed. These sorts of applications crop up from time to time in one’s career, and they present a unique set of challenges. I figured I’d store the results of my work on my blog, and if it helps someone else then great.

I am not posting the actual application here because it relies on some machine-specific hardware devices, but what I did instead was to extract the core functionality out into a sample application that shows how the duplex service is constructed and how it is used. I created a simple service that contains a list of Movies. The clients have the ability to add a new Movie to the server, or delete an existing one. There’s no fancy business rules or elaborate domain objects because I’m mostly interested in documenting the duplex service usage.

You can download the application here. It was written in C#, .Net 3.5. It utilizes StructureMap for dependency injection (which I highly recommend) and I’ve included that library with the package. The executable files are included in the corresponding /bin directories so you don’t have to compile the application if you want to see it work.

Notes:

  • The server is a console application. It will write to the console when a client subscribes or unsubscribes from the service, and it will print the count of the remaining subscribers.
  • To see the real-time updating in action, open two clients. Add a new movie title to the list in one client and you’ll see the immediate update in both clients.

WCF Duplex Service Contract

A note about this service: You will notice I am passing .Net objects across the wire with this service. There are no data transfer objects and no requirements for interoperability. I have complete control over both sides of the wire; the client and server applications. And in general, when you are writing a client-server app that’s going to be the case. This is not a “web services” solution; there are plenty of those already.

There is a caveat to creating and using a WCF Duplex service: You have to keep the connection open if you want your client to respond to callbacks. With WCF, once a connection falls into a Faulted state it becomes useless, and there’s no going re-connecting. You have to re-create the connection from scratch.

Normally, when using a one-way service, you only create the connection for as long as it takes you to call the service. But here we want our client to actively listen to callbacks from the service (as opposed to polling the service, which puts the burden for updates on the client and can leave the client with stale data). To make that happen we have to utilize a keepalive call with a timer, so that we ping the service frequently enough to maintain the connection. WCF will timeout a connection after 10 minutes of inactivity due to the way the underlying timeout scheme is written, and I have yet to find a single configuration knob that will allow a connection to live longer than that in a reliable fashion. The keepalive, however, works and is rock solid.

My service implements two interfaces. One is for the plumbing associated with subscribing and unsubscribing to the service, and using a keepalive to maintain the connection to the service. The second interface is the one the client is really interested in, and has all the calls to deal with the data. You don’t have to separate these interfaces; I did because it shields the client from having to see those plumbing methods (they can still get to them if they cast want to it).

The plumbing interface looks like this:

[ServiceContract(SessionMode = SessionMode.Required, CallbackContract = typeof (IMovieServiceCallback))]
public interface IDuplexService
{
    [OperationContract]
    bool KeepAlive();

    [OperationContract]
    void Subscribe();

    [OperationContract]
    void Unsubscribe();
}

The second service interface looks like this:

[ServiceContract(SessionMode = SessionMode.Required, CallbackContract = typeof(IMovieServiceCallback))]
public interface IMovieService : IDuplexService
{
    [OperationContract]
    IList<Movie> FindAll();

    [OperationContract]
    bool Exists(string title);

    [OperationContract]
    void Add(string title);

    [OperationContract]
    void Delete(Movie movie);
}

In both cases I’ve decorated the service interface with the [ServiceContract] attribute and specified the [CallbackContract], which looks like this:

public delegate void MoviesChangedEventHandler();

public interface IMovieServiceCallback
{
    event MoviesChangedEventHandler OnMoviesChanged;

    [OperationContract]
    void MoviesChanged();
}

The only thing that’s left to do on the client side is setup the configuration file where we specify the WCF settings and then implement the interface. I’m not going to post the XML here (you can download the sample app to see it, basic WCF stuff). What I will do is post the relevant code inside the service that deals with subscribers.

Service Implementation

The service maintains a list of subscribers which take the form of the IMovieServiceCallback. The three key methods for dealing with the subscribers are thus:

bool IDuplexService.KeepAlive()
{
    return true;
}

void IDuplexService.Subscribe()
{
    var callback = OperationContext.Current.GetCallbackChannel<IMovieServiceCallback>();
    if (!_subscribers.Contains(callback))
        _subscribers.Add(callback);

    RemoveDeadConnections();
}

void IDuplexService.Unsubscribe()
{
    var callback = OperationContext.Current.GetCallbackChannel<IMovieServiceCallback>();
    if (_subscribers.Contains(callback))
        _subscribers.Remove(callback);

    RemoveDeadConnections();
}

The KeepAlive is the simplest thing; just return a boolean. All we really care about is hitting the service to keep it open, and we do that with the aid of a timer in the client’s proxy (which we’ll get to later on).

Where things get interesting is how we handle the manipulation of data. In a one-way service the client would ask the service to create a new Movie and persist it. Once that operation was complete, the client would then proactively ask the service for an updated list of Movies. The client does all the work in this situation, making the requests. That’s fine for one client, but if other clients want to know when the data changes, they have to poll the service.

With a duplex service we tend to make a request of the service and then forget about it; the service will update us (and every other client) when its ready. So deleting a Movie looks like this:

public void Delete(Movie movie)
{
    _repository.Delete(movie);
    _movieCache = _repository.FindAll();
    UpdateSubscribers();
}

The service delegates to a Repository to handle the delete, updates its internal cache, and then makes a call out to all the subscribers, in essence telling them, “Hey, the data has changed. Update yourself”.

Updating the subscribers then becomes a matter of looping through them and calling the method on the callback interface:

private void UpdateSubscribers()
{
    for (int i = _subscribers.Count - 1; i >= 0; i--)
    {
        if (((ICommunicationObject) _subscribers[i]).State == CommunicationState.Opened)
        {
            _subscribers[i].MoviesChanged();
        }
        else
        {
            ((ICommunicationObject) _subscribers[i]).Abort();
            _subscribers.Remove(_subscribers[i]);
        }
    }
}

We are doing one extra step here: getting rid of delinquent subscribers. You see, there’s no easy, event-driven way to determine when a client has faulted or disconnected in a non-nice way. So the best plan is to periodically prune off bad subscribers. In my service, I do this whenever the data changes, and whenever a client subscribes or unsubscribes. It helps keep the subscriber list clean.

Client Proxy

With the service side of things mostly out of the way it’s time to move on to the client. Here I’ve manually created a client proxy (not using svcutil.exe). The proxy class implements the methods from IMovieService. Beyond the methods on the IMovieService interface, the proxy has a few responsibilities:

  1. Create the DuplexChannel and establish the connection to the service.
  2. Subscribe to the service.
  3. Create a Timer and fire the KeepAlive periodically.
  4. Unsubscribe from the service when shutting down.
  5. Respond to callback events from the server and re-fire them in a chain-of-command style pattern to the client.

Connect handles the first three responsibilities:

public void Connect()
{
            _movieServiceCallback = new MovieServiceCallback();
            _movieServiceChannelFactory = new DuplexChannelFactory<IMovieService>(_movieServiceCallback, _movieServiceEndpointConfigurationName);

            _movieService = _movieServiceChannelFactory.CreateChannel();
            ((IContextChannel) _movieService).OperationTimeout = new TimeSpan(0, 0, 0, 30);
            _movieService.Subscribe();
            _movieServiceCallback.OnMoviesChanged += MoviesChanged;

            _timer = new Timer(60000);
            _timer.Elapsed += TimerElapsed;
            _timer.AutoReset = true;
            _timer.Start();
}

A Disconnect method handles the fourth responsibility. Responding to the service is handled by the MovieChanged event, which simply re-fires it:

 private void MoviesChanged()
{
    if (OnMoviesChanged != null)
        OnMoviesChanged();
}

The other thing that I have the proxy do is make asynchronous calls to the service for methods that I know should be asynchronous. An example is Delete():

private delegate void DeleteDelegate(Movie movie);

public void Delete(Movie movie)
{
    DeleteDelegate deleteDelegate = _movieService.Delete;
    AsyncCallback callback = AsyncCompleteMethod;
    deleteDelegate.BeginInvoke(movie, callback, null);
}

private void AsyncCompleteMethod(IAsyncResult ar)
{
}

When the client makes a call to delete, it’s made asynchronously, and the client can immediately return. The UI doesn’t lock up and life goes on for the client. When the service is done processing the delete it will update the subscribers. That’s when they’ll have to respond in a way that doesn’t lock the UI or crash the program.

Client

In my view’s presenter class I take a dependency on the IMovieService interface. By way of StructureMap, the presenter gets the MovieServiceProxy, and we wire up to it’s OnMoviesChanged event:

private IMovieService _movieService;

public ShellPresenter(IMovieService movieService)
{
    _movieService = movieService;
}

public override void OnViewReady()
{
    base.OnViewReady();
    ((IMovieServiceProxy) _movieService).OnMoviesChanged += MoviesChanged;
    MoviesChanged();
}

The MoviesChanged method just queries the service for the list of Movies (which are conveniently cached in the service, so the service isn’t hitting the database every time a client requests the list).

private void MoviesChanged()
{
    View.Movies = _movieService.FindAll();
}

And this is where the trouble occurs if you don’t handle it correctly. We’ve just had what amounts to a background thread inform us to update our view. If we try and do this directly we’re going to get some unpleasant behavior in our UI. So we let the View handle this properly:

public delegate void UpdateMovieListDelegate(IList<Movie> movies);

public IList<Movie> Movies
{
    set
    {

        if (InvokeRequired)
        {
            var args = new object[1];
            args[0] = value;
            BeginInvoke(new UpdateMovieListDelegate(UpdateDataGridView), args);
         }
         else
         {
             UpdateDataGridView(value);
         }
     }
}

 private void UpdateDataGridView(IList<Movie> movies)
{
    _dataGridView.DataSource = null;
    _dataGridView.DataSource = movies;
}

InvokeRequired is a property on Controls, and it’s a convenient way to determine if the update is coming from another thread. If so, we can launch the operation asynchronously on the proper thread, in this case, the UI thread, since controls in .NET can only be updated by the thread they were created on.

Real-time, client-server type applications aren’t terribly difficult to deal with, they just require a different mindset in regards to when things get updated. And then one just has to utilize the asynchronous tools that the .NET framework has provided to make sure things get done in a way that doesn’t cause your application to hang or crash.

The Ruins

When I was first introduced to Dungeons & Dragons (second edition rules) I created a Ranger. I identified with that particular class because in real life I’m a bowhunter, and the two share many similarities. I thought the idea of a character that could survive in the wilderness, use a bow or swords effectively in combat, and had the ability to track creatures was a neat idea. When I played pencil and paper Dungeons & Dragons, I played rangers. Then the Baldur’s Gate and Icewind Dale games hit my computer, and it didn’t take long for me to realize that the ranger was a completely useless class.

Computer games have inherit limitations in them that make some of the more flavorful classes, like the ranger, more difficult to implement, and thus less attractive. In computer Dungeons & Dragons, rangers became nothing more than weak warriors. The skills that made them unique, like their ability to navigate the wilderness, or track game, couldn’t be properly implemented, and so they were removed from the computer games or severely weakened. This impacted the way I played those games, and thus I never created another ranger. Why play a weak ranger when you could play a strong warrior?

I’ve often wondered what a game might look and play like if it were to do justice to a ranger’s abilities. Now I know. Storm of Zehir is that game.

Storm of Zehir is an expansion to the Neverwinter Nights 2 game. It boasts two new important features:

  1. The ability to create an entire party of characters, a la Icewind Dale.
  2. An overland map that is used for travel, and which makes use of many ranger/druid skills.

Storm of Zehir leans heavily on these two new features. It succeeds because of them, but it is also dragged down by them. This is a game that doesn’t play nearly as well as it should.

For starters, while the game is billed as allowing you to create a full party of characters, similar to the Icewind Dale series of games, it doesn’t quite deliver on that promise. In Icewind Dale you could create a party of six characters. Six slots turns out to be just about the right size, allowing you to create a robust team of adventurers, complete with the necessary rogues, wizards, clerics and fighters. But Storm of Zehir limits your custom party size to four. Any experienced player will tell you that four just isn’t enough.

You are allowed one extra NPC follower. Two if you blow a precious feat slot on a “Leadership” ability. Unfortunately, unlike NPC characters in previous games, the NPC followers in Storm of Zehir are rather bland, and lack any fun or meaningful dialog. There exists no reason bring them into your party other than to fill a character slot, and most players are going to want to create their own characters anyway. Fortunately, there is a simple configuration fix that will allow you to create a full, six-character party. But that it something that should have been available out of the box.

Then there’s the overland map. This is Storm of Zehir’s greatest strength, and also its greatest weakness.

The overland map achieves something that no previous Dungeons & Dragons game has really been able to do. For the first time, abilities like Survival, Spot, Listen, and Move Silently are not confined to the realm of rogues and thieves. These skills are used extensively while your party is on the overland map. They allow your adventurers to avoid dangerous encounters, find hidden treasures, or discover important locations. Finally, it pays to have a Ranger in the group.

On its own merit, the overland map is a successful new feature. The problem with Storm of Zehir is not the overland map itself, but what the developers did with it. Or more precisely, what they did not do with it.

What they did not do is create a big, vibrant world. There are no large cities to frequent, no majestic ruins to explore, no labyrinthine dungeons to plunder. Everything in Storm of Zehir is small and absent of grandeur (or fun, for that matter).

The game plays very much like a Final Fantasy game. And that’s a shame, because the Neverwinter Nights 2 engine is capable of so much more. Like Final Fantasy, your party spends most of its time wandering the overland map, and when it finally does encounter a ruin, dungeon or random group of bandits, the encounter is small (a single zone), and is over in seconds. In fact, I have yet to encounter one dungeon that is larger than a single zone.

The cities are equally pointless. The main storyline sets your party up as a group of traders (a flaw in itself, since you should be adventuring, not establishing trade routes) and so the cities you encounter along the way are nothing more than trade points on the overland map. You can visit the pub or temple of a city directly, but that’s it. There’s no actual “city” to wander around in; there are no NPC’s to talk to, no quests to be had. The cities exist only to act as points of trade. Even the grand city of Neverwinter, a many-zoned metropolis from previous games, is limited to a single zone, and only a portion of that.

On top of the poor utilization of the overland map, Storm of Zehir turns out to be the most bug-ridden of the Neverwinter Nights games. It is overflowing with glitches and broken code, to the point that there is a 29-page post on the official forums full of complaints.

One of the biggest bugs you’re likely to encounter is watching your characters get stripped of their buffs each time they enter the overland map. This presents a problem when you run into your next random group of trolls or bandits, as you will only have enough time to cast one spell before your enemies are upon you. The four or five spell buffs your character might need to be efficient in combat will be missing. Fortunately, there is a player-created fix available. But this is such a huge flaw that it speaks to the quality (or lack thereof) on the Storm of Zehir development team. It’s the type of bug that should have been discovered in play testing and fixed long before the game came out.

The end result is a severely flawed game. Many of the new features, while welcome, are overshadowed by flaws in the game. The trading mechanism is dumb and boring; crafting was “simplified”, making it more of a money sink than anything else; the plot is thin; the cities are vacant entities; the overland map is relied upon too heavily; the encounters are too small and simple.

Storm of Zehir shows that it is not enough to create a cool, new game mechanism, but you have to actually utilize that mechanism to its fullest potential. Final Fantasy VII was released in 1994 and contained pretty much the same gameplay as Storm of Zehir. Hard to believe that after 14 years, we haven’t advanced the genre any further.

There is hope, however. The overland map gives us a glimmer of what is possible with this sort of game engine when used in conjunction with a lot of previously ignored character skills. Here’s hoping that if there ever is a Baldur’s Gate 3, or even another big expansion to Neverwinter Nights 2, that the developers learn their lessons from the failures of Storm of Zehir and build a better game next time. It would be a shame to have to put the Ranger back on the shelf.

Scott Fujita, via Peter King:

A lot of people think their vote won’t make a difference, or that their voice will never be heard. But if we all felt that way, then the system would be broken. There’s a lot more good than bad in this country, and the only way to begin to fix what’s bad is to get out and vote. Also, suffrage rights didn’t come without incredible struggle for a lot of people in this country — women and blacks in particular. To me, not voting would be a huge slap in their faces.

I can’t remember ever being so excited for an election day. This could be a landmark day in American history, and we’ll all be present to witness it. Provided, of course, the Republicans don’t cheat their way to victory once again.

Allow me to lay out a story that should sound familiar:

A couple of coworkers are discussing a bug in the software. In this particular bug’s case, it concerns an icon that appears in a column of a datagrid. Each row in the grid is either supposed to have this little icon in a specified column, or not. It depends on the data. But that’s not really important, just background info.

What’s really important is that the icon was previously coded and tested and worked. It appeared under the correct conditions and was hidden when it should be. Now, all of a sudden, it wasn’t working. We get an e-mail from customers saying, “The icon should show up in this circumstance, but it doesn’t.” Time to investigate.

While my two coworkers are hashing out exactly what could have happened, why, and how to fix it, I start thinking to myself, “There should be unit tests to catch this. This shouldn’t have happened. I’ll just go find the test suite and write a couple new tests to ensure the icon properly displays based on the criteria of the data.

Sounds easy, right?

As I started digging through the specific class code and examining the unit tests, I notice that there are no unit tests for this functionality. That seems suspicious. Then I notice something else: there’s no code. All that exists is a method call to a service that returns a datatable.

As Alice did, I decided to drop through the rabbit hole and see what’s up.

When I get to the bottom I see a gigantic SQL statement – several hundred lines long. It is embedded in the application, deep down in the data access layer, complete with multiple sub-selects and even an SQL case statement! It’s got multiple parameters in the where clause. It’s the kind of thing that makes SQL admins all warm and fuzzy inside, I’m sure. The whole thing is then handed off to our OR/M, which returns a simple datatable. While it is not a stored procedure, it might as well have been – it only has one step remaining on the cliff before it jumps right off into the database.

And the killer is, it’s untestable.

The caveat here is that this particular stored procedure was written as a performance gain, because the nature of the data requested is complex in its selection. But still, I’m of the belief that there might be a better way to query the data than one giant stored procedure.

Because the minute you put business logic in a stored procedure you compromise the quality of your application. You introduce the capability for your team to make mistakes that are no longer easily caught by a regression suite of unit tests. More importantly, I think, is that you increase the probability of repeated bugs. And to me, repeated bugs are a cardinal sin of programming. Nothing makes a development team look dumber than fixing a bug only to have it crop up again, and again, and again. It makes the software look bad, and by proxy it makes the developers look incompetent.