Easy XML Feed using LINQ to XML and ASP.Net MVC2

I recently had to create an XML feed for an app I’m working on.  The use case is simple.  I have a database of jobs and I need to provide an XML feed containing any jobs that were posted in the last week. I’m going to need to do this for several different 3rd party companies that each use a different XML format, so I want to do a little object oriented design to make sure that creating a feed using a new XML format is as easy as possible.

I’ve always heard that LINQ to XML makes working with XML much easier, and I know that MVC2 makes it easy to package up and return any type of data so I’m going to use them. It should make for a pretty painless XML feed.  Let’s see.

Starting at the controller

I want to start by writing the consuming code and let that drive the shape of my business layer code.  That means I start by writing a Feed action method on one of the controller classes in my MVC2 app.  I want to return XML data so i pick a return type of ContentResult, which will allow me to set the content type to “text/xml”.  The logic for this method is simple.  I have a feedKey that distinctly identifies the 3rd party company / XML schema that I want to generate a feed for.  All companies will hit the same action method, but they’ll pass in a different feedKey, which will result in them getting the appropriate XML feed.  My action method takes the feedKey as a parameter and then passes it to a factory method on my FeedService class that returns an instance of the appropriate FeedBuilder object.  The FeedBuilder is the class that I’ll use to encapsulate logic for getting job data and formatting it as an XML document. I’ll have a different FeedBuilder for each feedKey.  The design is a variation on the Strategy pattern.  So, back to controller action method. Here’s what it looks like.

        // Feed
        [AcceptVerbs(HttpVerbs.Get)]
        public ContentResult Feed(string feedKey)
        {
            var feedBuilder = this.FeedService.GetFeedBuilderFor(feedKey);
            return this.Content(feedBuilder.GetFeedXml(), "text/xml");
        }

The Factory Method

The factory method on my FeedService is pretty straight forward.  It just returns an instance of the right FeedBuilder for the feedKey that we pass in to it.  Note that the return type is FeedBuilderBase which is an abstract class that defines the shape of a FeedBuilder.

        // GetFeedBuilderFor
        public virtual FeedBuilderBase GetFeedBuilderFor(string feedKey)
        {
            switch(feedKey)
            {
                case FeedKey.SimplyHired:
                    return new SimplyHiredFeedBuilder();
                case FeedKey.Indeed:
                    return new IndeedFeedBuilder();
                default:
                    return new EmptyFeedBuilder();
            }
        }

The FeedBuilderBase

Remember we’re using a Strategy pattern.  Usually you would define the shape of your strategy class with an interface, but I’m going to use an abstract class because there’s some boilerplate code that will be the same for all of my concrete FeedBuilder classes and I want to implement that code in a single base class.  My abstract class is FeedBuilderBase and it looks like this.

    public abstract class FeedBuilderBase
    {
        //**************************************************************************************
        // PROPERTIES
        //**************************************************************************************
 
        // FeedService
        private FeedService _feedService;
        public virtual FeedService FeedService
        {
            get { return NewIfNull(_feedService); }
            set { _feedService = value; }
        }
 

        //**************************************************************************************
        // ABSTRACT METHODS
        //**************************************************************************************
 
        // GetFeedData
        public abstract List<JobFeedItem> GetFeedData();
  
        // BuildXmlFor
        public abstract XDocument BuildXmlFor(List<JobFeedItem> list);
 

        //**************************************************************************************
        // HELPER METHODS
        //**************************************************************************************
        
        // NewIfNull
        public virtual T NewIfNull<T>(T obj) where T : new()
        {
            if (obj == null) { obj = new T(); }
            return obj;
        }
  
        // GetFeedXml
        public virtual string GetFeedXml()
        {
            List<JobFeedItem> list = GetFeedData();
            XDocument xdoc = BuildXmlFor(list);
            var sb = new StringBuilder();
            var sw = new StringWriterUtf8(sb);
            xdoc.Save(sw);
            return sb.ToString();
        } 
    }

I have two abstract methods, GetFeedData and BuildXmlFor.  These methods represent the two things that change for each XML feed. Each company has slightly different business rules around what data they want to pull and each company has it’s own proprietary XML format.  These 2 methods will have to be overridden whenever we create a concrete FeedBuilder.   

You’ll also notice that we implemented GetFeedXml which is the method we called in our action method to get the text of our XML document. GetFeedXml should have been very easy to write, but it took a little time due to an unexpected encoding problem that I described in How to Create XML in C# with UTF-8 Encoding.

The Concrete FeedBuilder

Now I’m ready to create my first concrete FeedBuilder class.  I’m going to start with a jobsite aggregator called SimplyHired.  So I create a new SimplyHiredFeedBuilder class that inherits from FeedBuilderBase an I override my GetFeedData and BuildXmlFor methods.  GetFeedData is just a wrapper for the appropriate data access method in my FeedService class. GetFeedData returns a list of JobFeedItem objects.  JobFeedItem is a simple Data Transfer Object (DTO) that I created to contain the aggregate of data needed to build a feed item.

BuildXmlFor is more interesting.  It takes a list of JobFeedItem objects returned by our GetFeedData method and transforms it into an XDocument using a LINQ query.  The LINQ query is just a select with a bunch of nested XElement constructors that create the XML elements needed to represent each job.  We then take the expression created to represent the list of jobs as XElements and we wrap it in a single top level XElement called, wait for it…. jobs.  BTW, it is at this point that the LINQ expression actually executes.  Up to now it has just been an expression.  As soon as we use it in the constructor for our jobs XElement the expression compiles and executes.

So, we now have an XML tree contained in a single XElement called jobs.  To complete the method we just wrap the jobs XElement in a new XDocument, return the XDocument, and we’re done. 

    public class SimplyHiredFeedBuilder:FeedBuilderBase
    {
        //**************************************************************************************
        // FEEDBUILDERBASE OVERRIDES
        //**************************************************************************************
 
        // GetFeedData
        public override List<JobFeedItem> GetFeedData()
        {
            return this.FeedService.GetJobFeedItemsForSimplyHired();
        }
 
 
        // BuildXmlFor
        public override XDocument BuildXmlFor(List<JobFeedItem> list)
        {
            var xmlExpression = 
                from JobFeedItem j in list
                select new XElement("job",
                    new XElement("title", j.JobTitle),
                    new XElement("job-code", j.JobGuid),
                    new XElement("job-board-name", j.CompanyName),
                    new XElement("job-board-url", j.CompanyJobPageUrl),
                    new XElement("detail-url", ""),
                    new XElement("apply-url", GetApplyUrl(j.CompanyKey, j.JobGuid)),
                    new XElement("job-category", j.JobCategory),
                    new XElement("description",
                        new XElement("summary", j.JobDescription),                   
                        new XElement("required-skills", ""), 
                        new XElement("required-education", ""),
                        new XElement("required-experience", ""),
                        new XElement("full-time", j.IsFullTime),
                        new XElement("part-time", j.IsPartTime),
                        new XElement("flex-time", ""),
                        new XElement("internship", j.IsInternship),
                        new XElement("volunteer", ""),
                        new XElement("exempt", ""),
                        new XElement("contract", j.IsContract),
                        new XElement("permanent", j.IsPermanent),
                        new XElement("temporary", j.IsTemp),
                        new XElement("telecommute", "")
                        ),
                    new XElement("compensation",
                        new XElement("salary-range", ""),                   
                        new XElement("salary-amount", ""),
                        new XElement("salary-currency", ""),
                        new XElement("benefits", "")
                        ),
                    new XElement("posted-date", GetPostedOnDate(j)),
                    new XElement("close-date", GetClosedOnDate(j)),
                    new XElement("location",
                        new XElement("address", ""),  
                        new XElement("city", j.JobCity),  
                        new XElement("state", j.JobState),  
                        new XElement("zip", ""),  
                        new XElement("country", "") 
                        ),
                    new XElement("contact",
                        new XElement("name", j.ContactName),
                        new XElement("email", j.ContactEmail),
                        new XElement("hiring-manager-name", ""),
                        new XElement("hiring-manager-email", ""),
                        new XElement("phone", j.ContactPhone),
                        new XElement("fax", "") 
                        ),
                    new XElement("company",
                        new XElement("name", j.CompanyName),
                        new XElement("description", j.CompanyDescription),
                        new XElement("industry", ""),
                        new XElement("url", j.CompanyUrl)
                        )
                    );
            var jobs =  new XElement("jobs", xmlExpression);
            var declaration = new XDeclaration("1.0", "utf-8", null);
            return new XDocument(declaration, jobs);
        }     
    }

Conclusion

So I built it, tested it, and it works.  Except for the hour that I spend trying to figure out my UTF-8 encoding problem, the LINQ / XElement method was a relatively painless way to create XML. I love the simplicity of the MVC action method and the fact that it gives me a ContentResult type that gives me access the right parts of the stack.  The LINQ / XElement query was concise and the style of passing element data in XElement constructors makes it hard to mess up the document structure. My conclusion is that I like it and I’ll definitely be using this technique again.

How To Create XML in C# with UTF-8 Encoding

I can’t believe that I spent over an hour on this problem.  It’s one of those things that you can’t understand how it’s possible that it doesn’t just work. But it doesn’t, so here’s the solution I found for anyone who might have the same issue.

The Problem: My XML refuses to use UTF-8 encoding

I have some code where I’m doing a query and creating an XML doc from the data that’s returned.  I used LINQ to XML to create my XML and package it up in an XDocument.  The XDocument allows me to set the XML version and the encoding. Here’s the code I used to create and return my XDocument.

            var declaration = new XDeclaration("1.0", "utf-8", "yes");
            return new XDocument(declaration, jobs);

So far so good.  Now the last (and what should be trivial) step is to write that XML to a string and return it.  This is where things go wrong.  To get an XDocument object to write out a full XML document you call it’s Save method.  No problem, the Save method has an overload that takes a StringWriter parameter so I’ll just create a StringWriter and a StringBuilder and I’ll be in business.  Here’s my initial code.

        // GetFeedXML
        public virtual string GetFeedXML()
        {
            List<JobFeedItem> list = GetFeedData();
            XDocument xdoc = BuildXmlFor(list);
            var sb = new StringBuilder();
            var sw = new StringWriter(sb);
            xdoc.Save(sw);
            return sb.ToString();
        }

That all looks good. So I run it and here’s the result.

    <?xml version="1.0" encoding="utf-16" standalone="yes" ?>
    <jobs>
      <job>
        <title>Audit Engagement Manager</title>

What the heck is that???  UTF-16??  It’s like the UTF-8 encoding that I set in my XDocument was completely ignored and .Net decided to use UTF-16 encoding.  How can this not work?  Where else do I even have an option to set the encoding? The StringWriter won’t let me set the encoding, the StringBuilder certainly won’t let me set the encoding.

So I started Googling and found that other people were having the same problem and I found several solutions that sounded plausible.  Most of them centered around creating an XMLWriter with a settings object that would allow me to set the encoding.  All of these required a fair bit of extra code and I never got one of them to actually work, but they did get me looking at the StringWriter as the place where I needed to set my encoding.

The Solution: Subclass StringWriter

At this point I’m looking into progressively more and more complex solutions when I stumble across a post by Ian Dykes called Writing XML with UTF-8 Encoding using XmlTextWriter and StringWriter.  Ian basically says to create a new StringWriterWithEncoding class that inherits from StringWriter but allows you to set the encoding in the constructor.  I used the same idea and created the StringWriterUtf8 class below.  Instead of taking the encoding in a constructor, I opted to make the Encoding property always return UTF8.

    public class StringWriterUtf8 : StringWriter
    {
        public StringWriterUtf8(StringBuilder sb) : base(sb)
        {
        }
 
        public override Encoding Encoding
        {
            get { return Encoding.UTF8; }
        }
    }

Now I just need to use StringWriterUtf8 in my code instead of StringWriter and I’ll be using UTF-8 encoding in my writer.  I did it, I tested it, and it worked.  My XML output now looks like this:

    <?xml version="1.0" encoding="utf-8" standalone="yes" ?>
    <jobs>
      <job>
        <title>Audit Engagement Manager</title>

Thank you Ian for understanding more about this than I do and taking the time to put the solution out on your blog.  By the way, I think it’s worth mentioning that several people posted more “settings type” solutions in the comments to Ian’s post.  I tried them.  They didn’t work for me.  The only thing that did work was Ian’s idea of subclassing the StringWriter.  So I now have a working method and here’s the final version of GetFeedXml that uses my StringWriterUtf8 class.

        // GetFeedXML
        public virtual string GetFeedXML()
        {
            List<JobFeedItem> list = GetFeedData();
            XDocument xdoc = BuildXmlFor(list);
            var sb = new StringBuilder();
            var sw = new StringWriterUtf8(sb);
            xdoc.Save(sw);
            return sb.ToString();
        }

Refactoring a TrySave Method

When writing code, it’s unusual for me to get it right the first time.  Typically I’ll just write a big block of statements that do something that I’ll need done.  Then I’ll verify that it works.  Then I’ll look at it, notice what an ugly mess it is, and refactor it so that the poor developer who inherits this code (and that developer might very well be me) will have some hope of maintaining it. Here’s a typical example that came up this week.

The scenario

I’m working in a controller class in an ASP.Net MVC2 application.  I have some action methods that need to save the data passed in via a signup form.  The  signup form collects all of the data needed to create a new company and a new user in my application.  So my signup form is a composite of Company data, User data, an other data that’s needed for the UI (like a ConfirmPassword field).  I’ve chosen to encapsulate all of this stuff that the signup form needs in a view model class called SignupVM.  It looks like this.

    public class SignupVM
    {
        public User User { get; set; }
        public Company Company { get; set; }
        public string ConfirmPassword { get; set; }
        public string FormError { get; set; }
        public string FormMessage { get; set; }
 
        public SignupVM()
        {
            this.User = new User();
            this.Company = new Company();
            this.ConfirmPassword = string.Empty;
            this.FormError = string.Empty;
            this.FormMessage = string.Empty;
        }
    }

So that part works really well.  Here’s where it starts to get a little messy.  I have a couple of Action Methods in my controller that receive a SignupVM that’s been populated with data by the View (or more accurately, the ModelBinder) and they need to save that data.  Saving data involves three main steps.  First I need to validate the Company, the User, and my ConfirmPassword field.  Second I need to actually save the data, but I need to save the Company first and then make sure I set the correct CompanyGuid on the User before I save the User data. Third, I have some standard data that needs to get created for all new signups.  Right now the only standard data is the GeneralApplicantsJob but I know that there will be a need for more standard data down the road.  To handle this stuff, I created a TrySave() method in my controller class that looks like this.

        // TrySave - SignupVM
        private bool TrySave(SignupVM vm)
        {
            var userValidator = new UserValidator();
            var companyValidator = new CompanyValidator();
            var isError = false;
            // validate user
            try
            {
                userValidator.Validate(vm.User);
                ValidateConfirmPassword(vm);
            }
            catch (RuleException e)
            {
                isError = true;
                e.ErrorList.SetUiFieldNames(GetUserFieldNameMappings());
                e.ErrorList.CopyToModelState(ModelState, "User");
            }
            // validate company
            try
            {
                companyValidator.Validate(vm.Company);
            }
            catch (RuleException e)
            {
                isError = true;
                e.ErrorList.SetUiFieldNames(GetCompanyFieldNameMappings());
                e.ErrorList.CopyToModelState(ModelState, "Company");
            }
            // validate vm level fields
            try
            {
                ValidateConfirmPassword(vm);
            }
            catch (RuleException e)
            {
                isError = true;
                e.ErrorList.CopyToModelState(ModelState, "");
            }
            // if error, return false
            if (isError) { return false; }
            // if no error let's save the Company
            this.CompanyService.Save(vm.Company);
            // set the companyguid for the user
            vm.User.CompanyGuid = vm.Company.CompanyGuid;
            //save the user
            this.UserService.Save(vm.User);
            // create the GeneralApplicants job, it's created the first time we get it.            
            this.JobService.GetGeneralApplicantsJobForCompany(vm.Company.CompanyGuid);
            return true;
        }

Let’s Refactor

So, this block of code isn’t unmanageable, but I could definitely see it becoming unmanageable if I leave it like this and it gets added to a few times.  Plus, it’s not immediately clear what this code is doing.  What am I returning? Where am I returning it?  What logic decides what my return value is? Looking at it I realize that I’m writing in a very Imperative style that puts the focus on the individual steps of “how” to do something, instead of writing in a Declarative (or Functional) style that puts the focus on “what” is being done.  The code above is just a big list of steps that are a little difficult to follow without the comments.  I’m just validating data and saving data, so why do I have all of these statements in my method?  I can do better.

First let’s extract all of the validation code into an IsValidSignup() method.  Just that one refactoring helps quite a bit.  Now our TrySave method looks like this.

        // TrySave - SignupVM
        private bool TrySave(SignupVM vm)
        {
            if (!IsValidSignup(vm)) { return false; }
            // if no error let's save the Company            
            this.CompanyService.Save(vm.Company);            
            // set the companyguid for the user            
            vm.User.CompanyGuid = vm.Company.CompanyGuid;            
            //save the user            
            this.UserService.Save(vm.User);
            // create the GeneralApplicants job, it's created the first time we get it.            
            this.JobService.GetGeneralApplicantsJobForCompany(vm.Company.CompanyGuid);
            return true;        
        }

The next step is to refactor the data saving code.  There’s two things that bother me about this code.  First it’s critical that the the entities (Company and User) are saved it the right order, and that the user.CompanyGuid needs to be set in between the saves.  Second, it’s critical that the GeneralApplicantsJobForCompany get created after the Company has been saved, plus I know that there will be a need for more standard data later on, which means at some point a developer is going to have to modify this helper method in a controller class to make sure signups are created with the right standard data.  That feels like something that should be in my business layer, not in my UI code.  So, I decided to extract all of the persistence logic (including the creation of standard data) to a new method on my SystemService class in my business layer. The mehtod is called SignupNewCompany() and now my TrySave method looks like this.

        // TrySave - SignupVM
        private bool TrySave(SignupVM vm)
        {
            if (!IsValidSignup(vm)) { return false; }
            this.SystemService.SignupNewCompany(vm.Company, vm.User);
            return true;
        }

This looks much cleaner to me.  The focus is on what I’m doing instead of how I’m doing it, and the intent of the code is much more clear even though it no longer has comments.  Now I do have three methods instead of one, but each of those three methods does one thing that is easy to understand at a glance.  Best of all, I realized that I had put some logic in my UI that really needed to live in my business layer where it would be easier to reuse and easier to maintain.  For anyone who’s interested, here are all 3 methods in their final form.

 

        // TrySave - SignupVM
        private bool TrySave(SignupVM vm)
        {
            if (!IsValidSignup(vm)) { return false; }
            this.SystemService.SignupNewCompany(vm.Company, vm.User);
            return true;
        }

        // IsValidSignup
        public bool IsValidSignup(SignupVM vm)
        {
            var userValidator = new UserValidator();
            var companyValidator = new CompanyValidator();
            var isError = false;
            // validate user
            try
            {
                userValidator.Validate(vm.User);
            }
            catch (RuleException e)
            {
                isError = true;
                e.ErrorList.SetUiFieldNames(GetUserFieldNameMappings());
                e.ErrorList.CopyToModelState(ModelState, "User");
            }
            // validate company
            try
            {
                companyValidator.Validate(vm.Company);
            }
            catch (RuleException e)
            {
                isError = true;
                e.ErrorList.SetUiFieldNames(GetCompanyFieldNameMappings());
                e.ErrorList.CopyToModelState(ModelState, "Company");
            }
            // validate vm level fields
            try
            {
                ValidateConfirmPassword(vm);
            }
            catch (RuleException e)
            {
                isError = true;
                e.ErrorList.CopyToModelState(ModelState, "");
            }
            return !isError;
        }

        public void SignupNewCompany( Company newCompany, User newUser)
        {
            // create the new company
            this.CompanyService.Insert(newCompany);
            // create the new user
            newUser.CompanyGuid = newCompany.CompanyGuid;
            this.UserService.Save(newUser);
            // create the GeneralApplicants job, it's created the first time we get it.            
            this.JobService.GetGeneralApplicantsJobForCompany(newCompany.CompanyGuid);
        }