Category Archives: developing

Problems using the ‘sharedUserCertificates’ capability in the Windows Phone 8.1 Developer Preview

Windows Phone 8.1 introduces the long awaited support for certificate management. From the very first version of Windows Phone 7 it was possible to install certificates to the phone, e.g., by downloading them using Internet Explorer or by opening them from an e-mail message. But once installed, the only way to view and/or remove those certificates was to wipe your phone. Not very convenient.

As you may know from my earlier articles I did some sniffing of Windows Phone traffic using Fiddler. To enable the sniffing of SSL-traffic you need to install a Fiddler-generated root certificate, the infamous DO_NOT_TRUST_FiddlerRoot, to the phone. This enables Fiddler to perform man-in-the-middle attacks against the encrypted SSL connection.

Fiddler Root CA

Yesterday I tried to remove the Fiddler certificate from my phone, but was surprised I couldn’t find a certificate management settings app anywhere on the phone. As it turns out the Windows Phone 8.1 Developer Preview doesn’t come with a built-in certificate management app! Yes, the OS does now support certificate management, but only through the use of MDM software (i.e. Mobile Iron or Microsoft’s Intune/SCCM) or custom apps using the new certificate API’s.

Hey JoeB, was it that hard to add a simple Settings app for this? 🙂

Anyway, this gave me a good reason to check out those new certificate management API’s and I decided to write my own little certificate manager app. I inspected the Windows 8.1 ‘Cryptography and Certificate’ sample code and created a simple Windows Phone app using parts of that code (don’t you just love the converging of the Windows and Windows Phone API’s? 😉 ). To enable access to the certificate API’s I added the following capability to the package.appxmanifest:

<Capabilities>
    <Capability Name="sharedUserCertificates" />
</Capabilities>

Then I added the following lines of code (copied straight from said sample) to enumerate all certificates installed on the phone, so I could see if the Fiddler root certificate I had installed on the phone would show up:

private void Page_Loaded(object sender, RoutedEventArgs e)
{
    var task = CertificateStores.FindAllAsync();
    task.AsTask().Wait();
    var certlist = task.GetResults();
    Debug.WriteLine("Cert count: {0}", certlist.Count);
    LoadCertList(certlist);
}

private void LoadCertList(IReadOnlyList<Certificate> certificateList)
{
    CertListBox.Items.Clear();

    foreach (Certificate cert in certificateList)
    {
        CertListBox.Items.Add(cert.Subject);
    }
}

I fired up one of the Windows Phone emulators and installed the FiddlerRoot certificate by e-mailing it to myself and opening it from the e-mail message. Then I ran my test app. To my surprise I got zero results. I installed the FiddlerRoot cert again and reran my app. Again nothing… :-/

Well, maybe this was just emulator weirdness. So I deployed the app to a real phone, my trusty Lumia 920. This time another surprising thing happened. The deployment of the app failed and the following error message appeared in Visual Studio’s Output Window:

Error : DEP0001 : Unexpected Error: Package could not be registered. (Exception from HRESULT: 0x80073CF6)

I tried multiple times. I added some extra capabilities to the Package.appxmanifest, but it always failed with the above error message. A quick search turned up a StackOverflow topic of someone who experienced the exact same behavior. There Claus Jørgensen, a Skype developer, said the ‘sharedUserCertificates‘ capability was only available for first party developers (i.e. Microsoft, OEM’s and carriers).

So I contacted my buddy Matthijs Hoekstra, who happens to be a product marketing manager for Windows Phone and does a lot of the enterprisy stuff. Coincidentally, last week during the ‘Building Apps for Windows Phone 8.1 Jump Start’ he had talked about just this topic in his session ‘Windows Phone 8.1 for the Enterprise Developer’ and I was pretty sure this should be a supported scenario.

Building Apps for Window Phone 8.1 Jump Start Session 18

To cut a long story short: it’s a bug in the Windows Phone 8.1 Developer Preview!

Matthijs was able to confirm the behaviour. In fact, his Jump Start sample app (Session 18 Demos) also currently cannot be deployed to a real phone, as it also uses the 'sharedUserCertificates' capability.

Matthijs told me the ‘sharedUserCertificates‘ capability should be available to all Windows Phone developers, not just first party developers. 1 The fix for this bug will probably be included in the post-RTM updates that I expected to be released before General Availability of Windows Phone 8.1.

This still leaves me with the question why my app, when successfully deployed to the emulator, didn’t find any certificates. The code works fine in a Windows 8.1 WinRT app. Maybe it’s related to the deployment bug. Or maybe the phone is more restrictive about giving access to root certificates that haven’t been installed by the querying app itself? Heck, I may even be using the wrong piece of code 😀 Anyway,I’ll look into it. So as soon as I’ve got the answer, I’ll update this article.

1 Note that apps that require the ‘sharedUserCertificates‘ capability can only be deployed by sideloading or through a MDM solution. If you want to release such an app through the Windows Phone Store, you need special permission from Microsoft.

Prerequisite components error when creating a Visual Studio 2012 SharePoint 2013 App project

Yeah, I know… it’s a horrible blog title. But I hope Google/Bing/whatever brings you here in case you experience this error.

So you’re ready for doing some Office 2013 / SharePoint 2013 App development and happily installed Visual Studio 2012 and the Microsoft Office Developer Tools for Visual Studio 2012. You then quickly fire up Visual Studio 2012 (remember: using administrative privileges!), select the new “App for SharePoint 2013” template, work your way through the App for SharePoint settings wizard and finally press “Finish”….

BANG! The following error dialog appears:

An error occurred while trying to load some required components. Please ensure that the following prerequisite components are installed

For SEO purposes I will repeat the dialog text here:

“An error occurred while trying to load some required components. Please ensure that the following prerequisite components are installed:

Microsoft Web Developer Tools
Microsoft Exchange Web Services”

Very weird, especially since the Web Platform Installer should  have taken care of these two prerequisites. Since I encountered this error yesterday, I spent some time investigating it. I actually found some references to it here and here. As you can read in those threads the culprit seems to be a pre-installed instance of the Exchange Web Services API. I remembered having installed that API a week ago to do some testing. Apparently the Web Platform Installer doesn’t correctly handle a pre-installed version of the Exchange Web Services API very well.

The solution was simple: just uninstall BOTH the Exchange Web Services API and the Microsoft Office Developer Tools for Visual Studio 2012. Then reinstall (only) the Office Developer tools. This time the Web Platform Installer will ensure the correct Exchange Web Services API is also installed. If all went well you should be able to create a new “App for SharePoint 2013” project without any problems!

Building a SPQuery ViewFields string

If you’re querying SharePoint content using a CAML query from code it’s a good habit to always populate the SPQuery instance’s ViewFields property. Otherwise the returned SPListItem instances might not contain any data for certain fields and throw an exception when you try to access those fields.

Specifying ViewFields involves creating a string of CAML FieldRef elements. For instance if we want our query to return items that contain data for the Title, Created and ID field we use this:

   1: <FieldRef Name='Title'/><FieldRef Name='Created'/><FieldRef Name='ID'/>

As you can see there’s some overhead of boilerplate markup involved. I’ve written a small piece of code that I always use to make my life a little easier. Today I happened to post this code in a reply I wrote on the MSDN forums and also decided to submit it as Community Content to the official SPQuery docs on MSDN. Then I thought I might as well share it with you here. So here it is:

   1: public static string BuildViewFieldsXml(params string[] fieldNames)
   2: {
   3:     const string TEMPLATE = @"<FieldRef Name='{0:S}'/>";
   4:     StringBuilder sb = new StringBuilder();
   5:     foreach (string fieldName in fieldNames)
   6:     {
   7:         sb.AppendFormat(TEMPLATE, fieldName);
   8:     }
   9:     return sb.ToString();
  10: }
  11:  
  12: // Use it like this:
  13: SPQuery query = new SPQuery();
  14: query.ViewFields = BuildViewFieldsXml("Title", "Created", "ID");
  15:  
  16: // Note that you can specify a variable amount of string parameters, i.e.
  17: query.ViewFields = BuildViewFieldsXml("Title", "Created", "ID", "Author", "Gender");

Yeah, you’re right. This piece of code isn’t exactly rocket science. But you might appreciate it anyway 🙂

SharePoint Frustrations #1: The undocumented “IncludeTimeValue” CAML attribute

I’m planning on doing some posts about frustrating things I have encountered (and still do!) during my SharePoint development efforts. Here’s the first one:

Last year while working on a MOSS 2007 project for one of our customers I stumbled on what I thought was a bug in SharePoint 2007. I had created a custom list that was filled with Electronic Program Guide (EPG) information for the streaming video media that that site contained. I then created a Webpart that used ASP.Net AJAX to continually show the actual EPG information below the video stream.

In order to obtain the EPG items I used a CAML query to query the EPG list. The list was simply a custom list containing amongst others a column of type DateTime that was called "ProgramEnd". As the name suggests it contained the time the program ended. I then used the following code to create a query that was supposed to obtain the currently broadcasted item and all future items.

   1: const string EPG_QUERY_TEMPLATE = @"
   2: <Where>
   3:  <Geq>
   4:   <FieldRef Name='ProgramEnd' />
   5:   <Value Type='DateTime'>{0}</Value>
   6:  </Geq>
   7: </Where>"; 
   8:  
   9: SPQuery query = new SPQuery();
  10: query.Query = String.Format(EPG_QUERY_TEMPLATE,
  11:     SPUtility.CreateISO8601DateTimeFromSystemDateTime(DateTime.Now)); 
  12: // ...

To my surprise this query always returned too many items. Further investigation showed that time part of the query seemed to be ignored completely, so it returned the items as if no time was specified! So instead of getting the current item and all future items it returned all items broadcasted for that day.

After wasting a lot of time debugging, using the U2U Caml Query Builder and looking for an answer / solution on the internet I gave up and wrote a quick hack around it, which fortunately wasn’t very difficult. It would just run the query and run the results through some additional code that checked the ProgramEnd DateTime field and filter out the wrong items, like this:

   1: SPListItemCollection results = ...; // The results from the query mentioned above
   2: // Trim the results to only include the current and future items
   3: List<SPListItem> trimmedItems = new List<SPListItem>();
   4: foreach (SPListItem result in results)
   5: {
   6:     DateTime programEnd = (DateTime)result["ProgramEnd"];
   7:     if (programEnd >= DateTime.Now)
   8:         trimmedItems.Add(result);
   9: } 

Fortunately this worked just fine and the customer was happy. I was not… 😦

Today while surfing the Net I stumbled on this entry in the MSDN forums, from which I learned it wasn’t a bug, but that you need to included the "IncludeTimeValue" attribute to the CAML query, like this:

   1: <Where>
   2:  <Eq>
   3:   <FieldRef Name='programEnd' />
   4:   <Value Type='DateTime' IncludeTimeValue='TRUE'>
   5:    2008-07-30T12:00:00Z
   6:   </Value>
   7:  </Eq>
   8: </Where> 

If only I had known it was this simple… What bothers me is that this little, but very important attribute seems to be totally undocumented. I couldn’t find any information about it in the WSS / SharePoint SDKs.

Ofcourse once I knew what to look for I found some other blogs and forums mentioning this issue. It turns out the UCSharp blog had already blogged about this way back in October 2007, only a few months after I searched for it. Even Karine Bosch, U2U’s "CAML Girl" and author of the famous U2U CAML Query Builder, says she only recently found out about this. Fortunately she has included support for the "IncludeTimeValue" attribute in her latest version of the U2U Caml Query Builder, which I know a lot of SharePoint developers use to construct and test their CAML queries.

I also noticed that someone called puneetspeed has added some Community Content to the online SharePoint SDK’s SPQuery docs explaining this issue. So hopefully Microsoft will add information about the "IncludeTimeValue" attribute to the official SDK text in the near future.

Back from the DevDays 2008

Just arrived back from my visit to the Microsoft DevDays 2008 in Amsterdam. It was a great day and I’ve attended some interesting sessions.

I’ve uploaded the photos I took to my Flickr account, so check ’em out. I’ve also uploaded a small video of the “Holland Sport” bicycle race track we had at our booth. Check it out:

It was a very tiring day and I’m going to get some sleep now. I’ll blog about some more about the DevDays later on…

Do not reuse SPQuery!

Last week I was refactoring some of my SharePoint code. I stumbled on a loop that created a new SPQuery instance for each iteration. The code was something like this:

   1: SPList list = GetCommentsList();
   2: const string VIEWFIELDS = "";
   3:  
   4: foreach (string param in params)
   5: {
   6:     // Create fresh new SPQuery instance...
   7:     SPQuery query = new SPQuery();
   8:     query.ViewFields = VIEWFIELDS;
   9:     query.Query = BuildQuery(param);
  10:  
  11:     // And use it...
  12:     SPListItemCollection items = list.GetItems(query);
  13:     ProcessItems(items);
  14: }

As the value of the ViewFields property remained the same for each iteration I decided to create just one SPQuery instance and reuse it, like this:

   1: SPList list = GetCommentsList();
   2: const string VIEWFIELDS = "";
   3:  
   4: // Create just one SPQuery instance...
   5: SPQuery query = new SPQuery();
   6: query.ViewFields = VIEWFIELDS;
   7:  
   8: foreach (string param in params)
   9: {
  10:     // And reuse it...
  11:     query.Query = BuildQuery(param);
  12:     SPListItemCollection items = list.GetItems(query);
  13:     ProcessItems(items);
  14: }

To my surprise this code didn’t work! The first time the SPQuery instance was used it worked just fine. However, during the next iterations of the foreach loop it didn’t seem to get updated.

So, what have we learned today? Never reuse SPQuery instances!

Well… That’s not entirely true. You can reuse SPQuery instances for a very valid reason. If you use the RowLimit property you can limit the number of items returned in the query, which is useful for paging as seen in this sample code (taken from MSDN):

   1: using (SPWeb oWebsiteRoot = SPContext.Current.Site.RootWeb)
   2: {
   3:     SPList oList = oWebsiteRoot.Lists["Announcements"];
   4:     SPQuery oQuery = new SPQuery();
   5:     oQuery.RowLimit = 10;
   6:     int intIndex = 1;
   7:  
   8:     do
   9:     {
  10:         Console.WriteLine("Page: " + intIndex);
  11:         SPListItemCollection collListItems = oList.GetItems(oQuery);
  12:  
  13:         foreach(SPListItem oListItem in collListItems)
  14:         {
  15:             Console.WriteLine(oListItem["Title"]);
  16:         }
  17:         oQuery.ListItemCollectionPosition = 
  18:           collListItems.ListItemCollectionPosition;
  19:         intIndex++;
  20:     } while(oQuery.ListItemCollectionPosition != null);
  21: }

So, only reuse SPQuery instances if you use paging. If you change the actual CAML query you should create a new SPQuery instance for it.

.NET 3.5 Enhancements Training Kit

The Visual Studio & .NET Framework Evangelism team did it again. After their cool .NET 3.5 Training Kit they are now working on the follow up: the .NET 3.5 Enhancements Training Kit. It’s not final yet, but still very interesting!

Currently, the training kit contains six hands-on labs, made up of the following technologies:

  1. ADO.NET Data Services
  2. ADO.NET Entity Framework
  3. ASP.NET AJAX History
  4. ASP.NET Dynamic Data
  5. ASP.NET MVC
  6. ASP.NET Silverlight controls

Read more about it on Jonathan Carter’s blog.

Beware of the SharePoint Memory Leaks!

Beware of the SharePoint Memory Leaks!

You might not know it, but when developing for SharePoint (either WSS 3.0 or MOSS 2007) it is very easy to cause memory leaks, which can ultimately thrash your SharePoint server’s performance. The reason for this is that even though your own code might be written using only managed code, the SharePoint API still uses unmanaged code in some places. These unmanaged resources have to be explicitly disposed of, especially considering the fact they are living inside an ASP.Net web application (SharePoint) and potentially can have a long lifetime.

The problem is that the SharePoint API doesn’t always make clear when exactly you have to dispose of objects explicitly. Sometimes it may look like you’re just inspecting a value of a property, while in the background a whole new object is instantiated, which holds valuable unmanaged resources that should be freed by the caller (you!). Other times you might find yourself disposing an object, only to find out you weren’t supposed to…

Way back in June 2006 Microsoft employees Scott Harris and Mike Ammerlaan released their article Best Practices: Using Disposable Windows SharePoint Services Objects on MSDN, which can be regarded as the mother of all articles on this subject. I think this article is often overlooked and should be considered essential reading material for every serious SharePoint developer out there. Also read their more recent article called Best Practices: Common Coding Issues When Using the SharePoint Object Model, which also handles topics like data and object caching and writing scalable code.

Recently two other Microsoft employees also wrote interesting pieces on this subject. Stefan Goßner did a piece called Dealing with Memory Pressure problems in MOSS/WSS, in which he explains what a "Memory Pressure Situation" is and he lists some common causes and solutions for it. The other interesting read came from Roger Lamb. His entry is called SharePoint 2007 and WSS 3.0 Dispose Patterns by Example and in it he shows some very clear code samples and patterns on how to properly handle SharePoint objects.

Very cool material, guys!

Return of SmartPart V1.3 released

Yesterday Jan Tielens released a new version of his famous SmartPart to the community. It is now at version 1.3 and here’s the changelog:

  • Added a setup wizard to install the Return of SmartPart.
  • Added sample user controls (including connectable user controls and AJAX user controls).
  • Added localization support for ASP.NET AJAX user controls.
  • Various minor bug fixes.
  • Nice 2-minute screencast on how to deploy and test the SmartPart.
  • 64 bit version available.

Great work, Jan! I also like the fact you’ve used Lars Fastrup‘s SharePoint Solution Installer for deploying SmartPart.

Free e-book: The Developer Highway Code

Today I stumpled upon my TechEd 2007 goodie bag and found a little card I had picked up somewhere at the event. The card advertised a free prize draw (twenty copies of Vista Ultimate). All you had to do was download The Developer Highway Code, whatever that might be.

The Developer Highway Code Cover ImageOfcourse being Dutch means I’m always in for a free prize draw, so I immediately visited the URL printed on the card. It turned out The Developer Highway Code is a cool free e-book released by Microsoft (available in PDF and XPS format). It seems to have been around since 2006, but I hadn’t heard of it before. The book was written by Microsoft UK employee Paul Maher and Alex Mackman and has been revised in 2007 (probably right before TechEd). Here’s the official description, which you can find on the book’s website:

"To build software that meets your security objectives, you must integrate security activities into your software development lifecycle. This handbook captures and summarises the key security engineering activities that should be an integral part of your software development processes.

These security engineering activities have been developed by Microsoft patterns & practices to build on, refine and extend core lifecycle activities with a set of security-specific activities. These include identifying security objectives, applying design guidelines for security, threat modelling, security architecture and design reviews, security code reviews and security deployment reviews."

Unfortunately the free prize draw seems to be over, but the e-book is still available. I suggest every Microsoft developer to take a look at it, since it’s a very good summary of many things a developer should look at when developing secure applications.