The Darkening Sky: Prologue
A Neverwinter Nights 2 Campaign

The Darkening Sky: Necromancer Battle

Before I get into the details of this project, if you want to download it and play it, you can get the campaign in one of two places:

You can see some screenshots from my mod here.

The Journey to Amateur Video Game Modder

About a year ago I decided to fire up my old copy of Neverwinter Nights 2. For the uninitiated, Neverwinter Nights 2 is a computer video game based on the Advanced Dungeons & Dragons world and rules. I felt like playing in that world again; building a character, enjoying a story, running around slaying monsters and saving the world. It’s always a good time.

As I was playing the campaign that shipped with the game (known as the “O.C”, or “Original Campaign”, to the NWN2 community), I kept having thoughts like, “I’d do that scene differently“, or “I’d have done that encounter differently.”

I knew, from way back in 2002 when I first purchased the game, that NWN2 shipped with the toolset that the original developers used to generate the game content for the OC and subsequent expansion games. Everything you need to create your own story was inside that toolset: dialog editors, terrain editors, a quest journal, and even a very robust scripting engine. I am a software developer by trade; I figured I could do this. Certainly I would be comfortable scripting, and everything else I figured I could learn.

It was time to build my own video game.

This was something I had wanted to do since I compiled my first working program as an undergraduate in college. It is, after all, video games that lead me to my chosen profession. Way back in the late 1990′s I was an Electrical Engineering undergraduate. I spent the first two years of that profession taking general studies pre-requisites and math classes. When I got a taste of the real work in the EE field, however, I wasn’t interested in the subject at all. I was bored. I knew I didn’t want to be an electrical engineer and spend my entire working life wishing I had chosen a different profession.

Fortunately for me, at the University of Idaho, an introduction to computer programming course is a mandatory part of the EE curriculum. And as soon as I got into that class and started to learn about programming (with Turbo Pascal no less), I was hooked. I was good at it, for starters. I understood the principles immediately when others in the class were struggling to understand concepts like arrays and linked-lists, and it was (most importantly to me) fun. I enjoyed the work of writing code; I got a kick out of it.

Almost immediately I switched my major to Computer Science. I had found my calling. I had found something I enjoyed doing, and something I was good at.

And secretly, what I desired most in the world, what I knew programmers could do, was make video games.

My career arc never went in the direction of video game development (despite a close call), but the desire to build a game never left me. It’s something I have always yearned to do. And now was the time to do it. I just had to learn how to use the NWN2 toolset.

Back in 2002, when I first bought NWN2, I opened the toolset for a brief time. I had a burning desire to create my own campaign back then, just as I always had. But I immediately found the toolset daunting; it intimidated me, quite frankly. I was an inexperienced programmer and had not reached my potential yet; I didn’t have the confidence to tackle it. I quickly closed the toolset and decided that maybe I wasn’t cut out to build video games. Maybe I was just supposed to play them.

Then, last year, playing the game again, I got that itch; I wanted to build my own game and I knew the toolset would be a way to do it. The more I thought about it, the more I realized I could figure it out. Lots of people – many of them not programmers – have used the NWN2 toolset to make their own games and tell their own stories. If they could do it, why couldn’t I?

When I opened the NWN2 toolset last year, I realized that I could do it, and it wasn’t nearly as daunting as I thought it was. I started scanning the internet with Google, looking for tutorials, learning the toolset and figuring out how to build my dream game. Sure, it was going to take time, and there was a lot I needed to learn about using the toolset, but I was determined to do it. The programming and scripting aspects I knew I could do. I was less sure about my artistic abilities. After all, a large part of a video game is art and dialog. But I wasn’t going to let anything stop me. And so I fired up the toolset and started building a world; I started to tell my story.

The intent of The Darkening Sky was to build a game I wanted to play. I had no other audience in mind but myself. The story was something that had been sitting inside my head for many years of playing Dungeons and Dragons games. I went straight to work. During the day I wrote code for my real job. In the evenings and weekends I created The Darkening Sky…

The result is the “prologue” to the story. This is very much a “beginning” mod. I intend to keep working at it, building content, and releasing chapters until the story is complete. I have a definite beginning, middle and end in mind. Everything is written down, outlined, planned and ready to be built. Now it’s just about continuing to put in the time and effort to make it real.

This week I released the prologue officially on the Nexus and Vault 2.0 web sites. It’s been a really good feeling, to spend that much time on something and release it into the wild. People are downloading it and playing it. There’s a great sense of satisfaction in that.

But the story isn’t finished.

Time for me to get back to the toolset…

Wow, time flies. I realize I haven’t written a blog post in over 2 years. My last post was about Silverlight.

Silverlight.

Yeah… that’s just wrong.

Who uses Silverlight anymore? Anyone?

Shortly after I wrote that last post we switched to Microsoft’s MVC3 platform. And I was so happy we did. It was a huge improvement. Why didn’t I blogged about it? Because I’ve been too damn busy writing code. Being productive and all of that. Making clients happy. Working.

In my professional career, I cannot recall working on a technology stack that allowed me to be more productive. Granted, I don’t have the experience with Ruby on Rails, and some of the other “really fast” tech stacks. But this has been my career path, and this is what I’ve been exposed to. MVC is allowing us to just get stuff done. And that has been fun.

So what exactly have we been using?

  • .NET 4.0
  • MVC3
  • JQuery
  • Oracle
  • Home-grown IoC Container

I know, I know. It’s not NHibernate and SQL Server. It’s Oracle.

“Isn’t that painful?”, you ask.

No, not really.

I’ve been kind of surprise, really, about how my attitude toward Oracle has changed in the past couple of years. I came from a SQL Server shop. I liked it. I came from a place where I was able to use NHibernate a little bit. I liked that too.

In the beginning, Oracle was more painful. Getting data into and out of our database was a slower coding process. Quite often it was the slowest thing we had to do. But that all changed when we wrote a fluent-ish interface in .Net for dealing with Oracle. Now? It’s pretty damn easy…

“And what’s that home-grown IoC Container all about?”, you ask.

It’s about not having to rely on someone else’s code and having something we can control in-house.

Initially I brought StructureMap into our environment. I was the only developer in our group to really use a container, and I would champion that idea every time I had a chance. Eventually I sold the right people on the idea that we should be doing this in all our applications. But there was some disagreement about how to proceed. Not everyone liked the idea of a core part of our design being a 3rd party library that we didn’t control. So, in the end, we wrote our own. It’s a lightweight container and does only what we need it to do. It works. And it’s easy.

The end result of this tech stack is that we can spin up a project in minutes, and we can build apps in a much shorter time frame. I spend a lot less time now working on infrastructure code and a lot more time just writing application code. Life is good when your stack lets you do that.

One of the nice features of Silverlight with RIA services – one of the things I really dig – is how easy it is to perform data validation. This is how I have always imagined data validation should work. It’s simple and segregated and looks damn pretty on the screen. I don’t have to write any special code on the client to make the validation happen, I just call my service and tell it to persist my object and it lets my user know – via some very nice UI graphics – if there’s a problem with the data.

Silverlight ComboBox Validation

I can write complex server-side validation code that queries against a database or I can even write validation code that checks against combinations of properties on the object in question. This is good stuff.

That said, validating data that comes from a ComboBox can be a bit confusing, at least initially.

Validating something like a string property on an object is straightforward. You’re dealing with one property on the object and one simple binding (like a TextBox) on the client. If the data is invalid, you’re going to inform the Silverlight’s validation mechanism by way of a ValidationResult, telling it which property on the object was in error. Silverlight takes care of the rest.

But ComboBox data can be slightly less straight forward when you’re dealing with child objects that are referenced by the parent via a foreign key. Typically we’re talking about a lookup scenario – data that is stored in lookup tables and we’re trying to force the user to choose from a set of available values. Now you have two pieces of data on your parent object that matter: the child object itself and the foreign key.

Silverlight ComboBox Validation Diagram

Here we have two child objects: Genre and Studio. Each are associated to the parent object, Movie, by way of their keys. Also on the parent object are references to the actual child objects themselves. These objects are simple, but the reality is they can (and may) be much more complicated child objects.

 
public partial class Movie
  {
    public int MovieId { get; set; }
    public string Title { get; set; }
    public int YearReleased { get; set; }

    public int GenreId { get; set; }
    public Genre Genre { get; set; }
    public int StudioId { get; set; }
    public Studio Studio { get; set; }
  }

When you query for a Movie object from your service you’re going to get those child objects attached to the parent, and then you can use their data for databinding.

The tricky part here is that each piece of data (the child object and the key) is important for different reasons, and you’re going to use both. The foreign key is important because that’s what you want to actually validate against – you want to make sure the value actually exists in the database. We’re going to do that server-side with a validation class.

 
public class MovieValidator
  {
    public static ValidationResult ValidateHasGenre(int genreId, ValidationContext context)
    {
      var service = new MovieService();
      var genre = service.GetGenres().Where(x => x.GenreId == genreId).SingleOrDefault();

      if (genre  != null) return ValidationResult.Success;

      return new ValidationResult("Must select a Genre", new[] {"Genre"});
    }

    public static ValidationResult ValidateHasStudio(int studioId, ValidationContext context)
    {
      var service = new MovieService();
      var studio = service.GetStudios().Where(x => x.StudioId == studioId).SingleOrDefault();

      if (studio != null) return ValidationResult.Success;

      return new ValidationResult("Must select a Studio", new[] {"Studio"});
    }
  }

The reason we need to validate against the foreign key is because RIA will not send the actual child object back across the wire when you try and persist the parent object. If you check the values of the Genre and Studio properties on the Movie object when it gets sent to the server, you’ll notice they are both null. That’s because RIA knows that the only piece of information that’s actually important to the parent object is the key. This is by design so that RIA sends less data over the wire.

However, when we want to inform the client that the value is invalid, we don’t want to tell the client that the key is invalid because the key isn’t bound to anything on the screen. What we want to give Silverlight is the same name as the property that the ComboBox is bound to. In this case, the Studio or Genre properties.

 

This is why, when the validation fails, we send back the correct property name as the second parameter on our ValidationResult:

 
   return new ValidationResult("Must select a Genre", new[] {"Genre"});

By doing this, Silverlight will be able to find the appropriate binding on the UI and mark it as invalid. We get the pretty red border and mouseover text telling us that we have to choose something from the ComboBox.

This isn’t anything terribly difficult, it just may not be immediately intuitive the first time you try and validate the data and don’t get the expected red border around the ComboBox.

Source code can be found here

You can read Part Ihere.

In the last post I gave an overview of the sample Silverlight + RIA application (code can be found here). For Part 2, I wanted to talk more about the specifics of the service layer and the order of the calls.

You’ll notice the declaration of several data sources in the constructor of the service:

[EnableClientAccess()]
  public class TicketService : DomainService
  {
    private TicketDataSource _ticketDataSource;
    private CustomerDataSource _customerDataSource;
    private ProductDataSource _productDataSource;
    private ActionTypeDataSource _actionTypeDataSource;
    private ResultTypeDataSource _resultTypeDataSource;
    private TicketActionDataSource _ticketActionDataSource;

    public TicketService()
    {
      _ticketDataSource = new TicketDataSource();
      _customerDataSource = new CustomerDataSource();
      _productDataSource = new ProductDataSource();
      _actionTypeDataSource = new ActionTypeDataSource();
      _resultTypeDataSource = new ResultTypeDataSource();
      _ticketActionDataSource = new TicketActionDataSource();
    }

In our production environment we’re delegating the actual database operations to separate classes that make calls to Oracle Stored Procedures through ADO.NET using an Oracle provider. It doesn’t have to be an Oracle DB of course – you could use any database you like when you just want to hide the details away behind an abstraction such as this. For the sample application, we store the data in memory using the HTTP Session.

Insert: Order Of Operations

What I really wanted to talk about in this post is the order of operations when it comes to doing inserts and saving new entities. For this application we have three pieces of data that can be inserted into the database:

  • Ticket
  • Customer
  • TicketAction

Product, ActionType and ResultType are also data on the Ticket or TicketAction objects, but they are pulled from the database tables as static lookups and only referenced in their parent objects as foreign keys. No new values are inserted into these three tables via the application and that is why they do not have INSERT methods on the TicketService.

Ticket is our root aggregate object. Customer is a child object on the root, and TicketActions are a collection on the root. The order of operations is determined by RIA. InsertCustomer() will be called first, then InsertTicket() and finally, InsertTicketAction(). Let’s see how and why this happens.

Inserting a Customer

All single-entity child objects on the root will have their respective INSERT methods called first. If you think about it, this makes sense. The primary key of each child object is used as a foreign key on the parent object, so the child object primary key has to be known BEFORE the parent object can be saved. Thus, for a new Customer object, we have to first insert it into the DB and return the newly generated CustomerId to the Ticket BEFORE the Ticket gets saved.

public void Insert(Customer customer)
{      
    customer.CustomerId = _customerDataSource.Insert(customer);
}

So here we’re passing the Customer to a class that will perform the actual INSERT into Oracle, and in turn that method will get the generated PK from Oracle and return it. It is then our responsibility in this call to associate that PK with the Customer object, because the Customer is going to get sent back across the wire to the client when RIA is done making all these calls.

Inserting a Ticket

Once the Customer has been saved to the DB (and the new CustomerId returned and set on the Customer object) then RIA will call the INSERT method for the Ticket itself. At this point in time we need to set the CustomerId (and any other child foreign keys) on the Ticket object:

public void Insert(Ticket ticket)
{
      ticket.CustomerId = ticket.Customer.CustomerId;

      ticket.TicketId = _ticketDataSource.Insert(ticket);

      foreach (var ticketAction in ticket.TicketActions)
      {
        ticketAction.TicketId = ticket.TicketId;
      }
}

And just like the Customer INSERT method, once we save the Ticket to the database we need to get the generated PK back from Oracle and set it on our Ticket.

Inserting TicketActions

The last thing that happens here is the insertion of multiple TicketActions, as this is a child collection on the Ticket object. And again, if you think about it from a DB point of view, this makes sense. Each TicketAction has a foreign key TicketId. Thus, a TicketAction can’t be saved to the DB until the TicketId is known. For a new Ticket, that won’t be until the Ticket is saved and the id returned. Once the Ticket’s primary key (TicketId) is known, we can attach it to each TicketAction and save them as well. RIA will make that call automatically, after the INSERT method for the Ticket is done:

public void InsertTicketAction(TicketAction ticketAction)
{
      ticketAction.TicketActionId = _ticketActionDataSource.Insert(ticketAction);
}

Done Inserting

Once all of those calls are made and the various pieces of data have been saved to their respective tables, the root aggregate object is ready to be reloaded by RIA. This will happen AUTOMATICALLY. You do not need to call Load() on the service context again in your ViewModel after you have called SubmitChanges(). RIA will automatically reload your entity client-side depending on what kind of LoadBehavior you specified (for this project we’ve specified a LoadBehavior.RefreshCurrent). This is another reason why it is so important to return the primary keys and set those values on the objects when you saved that data to the database.

Next time I’ll cover UPDATE operations and explain one of the quirks of RIA sending entities down the wire….

There are a lot of Silverlight + RIA samples all over the internet and many of these will happily show you how to properly build functionality with these emerging technologies. Unfortunately, they all seem to be built on the notion that you will be using Microsoft’s Entity Framework with SQL Server for your database needs. Well, where I work, we are tied to an Oracle database and thus the Entity Framework is not really an option. So how could we make this work? I’m going to show you.

Disclaimer: I don’t proclaim to be a Silverlight or RIA expert. This is just a account of what we discovered when building applications with Silverlight and RIA Services against an Oracle database.

The source code for this (and related posts) can be found here. This is a Visual Studio 2010 Solution.

The Application

Silverlight Aggregate Application

This is a simple ticketing application example. Companies use these sorts of applications to track call and customer interactions. Operations you can perform:

  • Open a Ticket
  • Create a new Ticket
  • Search for a Customer when creating a new Ticket
  • Log a Ticket Action
  • View the History of Ticket Actions
  • Edit Customer data when saving a Ticket
  • Edit Product (via dropdown selection) on an existing Ticket

In our production environment we write these applications against an Oracle database. For the purposes of this demo, everything is stored in memory. But the data access part of the RIA Service is abstracted, so you should be able to see how easy it is to hand operations off to a data access layer to work against your preferred database.

Complex Aggregate Roots

To start with, many of the samples for Silverlight and RIA on the internet are very simple. They show basic entities in relatively simple scenarios. A lot of the “magic” happens by just calling on the Entity Framework inside the RIA Services context. We all know by now that’s not the way the real world works, and the first time you try and do something outside the scope of a demo you run into headaches.

What I am going to show you here is a complex root aggregate object that is actually structurally identical to one we used in our real production application. I’ve trimmed it down some (removed a few entities for brevity), but the structure is the same. The core pieces to this aggregate are such:

  • A root parent object
  • Multiple child objects contained in the root
  • A List of child objects
  • Child objects on the list of child objects

Does that sound more like a real world scenario to you? Good, me too. Here’s an object graph to help visualize:

Root Aggregate

Each child object represents a table in an Oracle database. Our goal with our RIA service is to be able to query the child objects (Product, Customer, TicketAction, ActionType, ResultType) separately (so we can fill in dropdowns and forms) and to be able to query and save the root aggregate (Ticket) as an entire object graph all at once. To make this happen, we have to do two things.

First, we have to setup the metadata correctly so that the associations between the objects can be understood by RIA, because RIA is going to do a lot of magic for us. Second, we have to develop the code inside our RIA Service methods properly so that data gets inserted/updated in the proper order when we’re saving an object graph.

Metadata & Foreign Keys

In a typical database scenario, you’re going to have a parent table that will contain the foreign keys of the child tables. However, when you want to actually work with this data in code, you don’t want just the keys to those children showing up in your root object/entity – you want the whole child object/entity. When using an O/RM this sort of thing is handled for us automatically. But when you’re using more primitive database access technologies (ADO.NET with Oracle) then you have to account for these facts yourself. With RIA Services we can make this happen, but we have to provide the parent object with both pieces of data: the foreign key to the child object AND the child object itself.

  [MetadataType(typeof (TicketMetadata))]
  public partial class Ticket
  {
    internal sealed class TicketMetadata
    {
      [Key] public int TicketId;

      [Required]
      public DateTime IncidentDate;

      public int CustomerId;

      [Required]
      [Range(1, 99999999, ErrorMessage = "Must select a Product")]
      public int ProductId; 

      [Include]
      [Association("Ticket_Customer", "CustomerId", "CustomerId", IsForeignKey = true)]
      public Customer Customer;

      [Include]
      [Association("Ticket_Product", "ProductId", "ProductId", IsForeignKey = true)]
      public Product Product;

      [Include]
      [Composition]
      [Association("Ticket_TicketActions", "TicketId", "TicketId")]
      public List TicketActions;
    }
  }

As you can see, we have the ProductId and Product as properties on the Ticket (same with CustomerId and Customer). We instruct RIA to associate these two things via this line:

[Include]
[Association("Ticket_Product", "ProductId", "ProductId", IsForeignKey = true)]
public Product Product;

The important part here is to set the IsForeignKey = true. RIA Services will use this to association to copy key values among related entities. So, for instance, when you set a Product with a ProductID = 3 on the Ticket (parent) object, RIA’s proxy class for Ticket knows to automatically set the Ticket.ProductId = 3.

Collections

Collections are handled slightly differently from individual child objects. There are a couple of ways to deal with them, and your choices depends on whether or not you want your child objects in the collection to be accessible as separate methods on the RIA Service, or only through the parent object. In our case, we wanted these particular child objects to only be accessible via the parent object, so we utilize the Composition attribute. The Composition attribute is well documented. The highlights:

  • Hierarchical change tracking – when a child entity is modified, it’s parent also transitions to the Modified state.
  • When a parent is in the Modified state, all of its children are included in the change-set sent to the server, including any Unmodified children.
  • Operation ordering – Often only CRUD operations for the parent or root type in a compositional hierarchy will be exposed by a DomainService. This allows you to write all your business logic for a hierarchy in a single method. However writing explicit methods for child Types is supported, in which case parent operations are always called before child operations. For example, if a new OrderDetail was added to an existing PurchaseOrder, the Update method for PurchaseOrder would be called before the Insert for the OrderDetail.
  • Public EntitySets for child Types are not generated on the code-genned DomainContext. Children are only accessible via their parent relationship.

Take note of the third item: parent operations are always called before child operations. When you’re writing RIA Services, order of operations is important. As you begin to develop your data access code (whether it’s Oracle or some other DB) it is important to pay attention to the order of the calls as RIA makes them. Returning surrogate keys from INSERT operations will be important when dealing with parent and child data.

The RIA Service

The nuts and bolts of this application lies in the service layer. I’m going to post the code so you can see how simple it is. In the next post, I’ll talk about order of operations – how the calls get made by RIA and in what order, and why you have to do things a certain way.

  [EnableClientAccess()]
  public class TicketService : DomainService
  {
    private TicketDataSource _ticketDataSource;
    private CustomerDataSource _customerDataSource;
    private ProductDataSource _productDataSource;
    private ActionTypeDataSource _actionTypeDataSource;
    private ResultTypeDataSource _resultTypeDataSource;
    private TicketActionDataSource _ticketActionDataSource;

    public TicketService()
    {
      _ticketDataSource = new TicketDataSource();
      _customerDataSource = new CustomerDataSource();
      _productDataSource = new ProductDataSource();
      _actionTypeDataSource = new ActionTypeDataSource();
      _resultTypeDataSource = new ResultTypeDataSource();
      _ticketActionDataSource = new TicketActionDataSource();
    }

    public IQueryable GetTicketById(int id)
    {
      var tickets = _ticketDataSource.GetTickets();
      return tickets.Where(x => x.TicketId == id).AsQueryable();
    }

    public IQueryable GetTickets()
    {
      return _ticketDataSource.GetTickets().AsQueryable();
    }

    public void Insert(Ticket ticket)
    {
      if (ticket.CustomerId == 0)
        ticket.CustomerId = ticket.Customer.CustomerId;

      ticket.TicketId = _ticketDataSource.Insert(ticket);

      foreach (var ticketAction in ticket.TicketActions)
      {
        ticketAction.TicketId = ticket.TicketId;
      }
    }

    public void Update(Ticket ticket)
    {
      foreach (var ticketAction in ticket.TicketActions)
      {
        if (ticketAction.IsNew)
          ticketAction.TicketId = ticket.TicketId;
      }

      _ticketDataSource.Update(ticket);
    }

    public void InsertTicketAction(TicketAction ticketAction)
    {
      ticketAction.TicketActionId = _ticketActionDataSource.Insert(ticketAction);
    }

    public IQueryable GetProducts()
    {
      return _productDataSource.GetProducts().AsQueryable();
    }

    public IQueryable GetActionTypes()
    {
      return _actionTypeDataSource.GetActionTypes().AsQueryable();
    }

    public IQueryable GetResultTypes()
    {
      return _resultTypeDataSource.GetResultTypes().AsQueryable();
    }

    public IQueryable GetCustomers()
    {
      return _customerDataSource.GetCustomers().AsQueryable();
    }

    public IQueryable GetCustomersBySearchCriteria(string firstName, string lastName, string phone)
    {
      return _customerDataSource.GetCustomersBySearchCriteria(firstName, lastName, phone).AsQueryable();
    }

    public void Insert(Customer customer)
    {
      _customerDataSource.Insert(customer);
    }

    public void Update(Customer customer)
    {
      _customerDataSource.Update(customer);
    }
  }

This post is already long. Part 2 coming soon!