Nifty Code: Output Azure Blob Directory As Zip File

Following the release of several new Azure features, I’ve been tinkering again with the various cloud offerings.  I’m very impressed with this release, which brings ‘Virtual Machines’ for super-cheap, super-easy Virtual Private Server capability.  In addition, the new ‘Websites’ model makes developing for Azure as easy as any other hosting provider – Git, FTP, TFS, and Web Publishing support make it super simple to get code up into the cloud, where you have buttery-smooth configuration and monitoring capabilities.

In the course of my tinkering, though, I whipped up a nifty little code snippet I thought I’d share.  Say you’d like to let a user download a zip archive of files and folders stored in Azure Storage from an ASP.NET MVC or Web Forms app. Thanks to the de-facto standard .NET Zip library, SharpZipLib, this is relatively easy:

       private void OutputBlobDirectoryAsZipDownload(CloudBlobDirectory directory)
        {
          
            byte[] buffer = new byte[4096];
            Response.ContentType = "application/zip";
            var fileName = directory.Uri.Segments.Last().Replace("/", "");

            Response.AppendHeader("content-disposition", "attachment; filename=\"" + fileName + ".zip\"");
            Response.CacheControl = "Private";
            Response.Cache.SetExpires(DateTime.Now.AddMinutes(3)); 
            ZipOutputStream zipOutputStream = new ZipOutputStream(Response.OutputStream);
            zipOutputStream.SetLevel(3); //0-9, 9 being the highest level of compression
            var allFiles = directory.ListBlobs(new BlobRequestOptions { UseFlatBlobListing = true }).Where(x => x.GetType() == typeof(CloudBlockBlob)).Cast<CloudBlob>();

            foreach (var file in allFiles)
            {

                Stream fs = file.OpenRead();    
                var entryName = file.Uri.ToString().Replace(directory.Uri.ToString(), "");
                ZipEntry entry = new ZipEntry(ZipEntry.CleanName(entryName));
                entry.Size = fs.Length;
              
                zipOutputStream.PutNextEntry(entry);

                int count = fs.Read(buffer, 0, buffer.Length);
                while (count > 0)
                {
                    zipOutputStream.Write(buffer, 0, count);
                    count = fs.Read(buffer, 0, buffer.Length);
                    if (!Response.IsClientConnected)
                    {
                        break;
                    }
                    Response.Flush();
                }
                fs.Close();
            }
            zipOutputStream.Close();
            Response.Flush();
            Response.End();
        }

This is essentially just their sample code adapted to work with Microsoft’s Azure StorageClient code.  To use, simply use NuGet to add SharpZipLib and StorageClient to your app – it need not be an Azure application.  Then call this code, passing in a CloudDirectory instance, and viola! Cloud files packaged in a single zip file for your users to download.

There are some caveats: there is an upper limit to the number of files the zip can contain, and likely in the size of the zip.  I’m not sure this would work for downloading gigs of data. But, I can think of a few scenarios where it might be interesting. 

Blackbelt SharePoint Debugging

You know the 14 hive like the back of your hand, and can lookup a correlation id in 10 seconds flat.  You know how to handle everything Health Analyzer can throw at you.  Central Admin bores you – you like the cozy black and white of the PowerShell console.  And now, some third party thing is broken, not logging, and all tech support can do is ask you to reboot the servers (which you did 3 days before you called support- FizzBin) 

You have come a long way – it’s time to complete your training and become a SharePoint Debugging Blackbelt.  You will be armed with three tools:

• dotPeek – a free .NET decompiler from JetBrains.  Yes, we’re going there.
• PowerShell – this is just .NET in the command line.  You’re about to kick some SharePoint butt.
• A working knowledge of SharePoint and .NET.

Rule #1: Know your Enemy

dotPeek allows you to decompile and look at the code of any managed .NET assembly.  Including those in the GAC, and those written by Microsoft.  SharePoint is, in the end, really just a huge ASP.NET app.  Almost all SharePoint 3rd party add-ins are written in .NET.  You see where I’m going here.

Isolate the problem.  Do your best to get repeatable steps and know more or less what part of SharePoint or 3rd party is not working as expected.  If the problem is on a particular page under _layouts, note the page, and then open it from the corresponding folder in the 14 hive.  At the top, especially if it is a 3rd party, note the namespaces and assemblies involved.  For example, the Page directive may have a “Inherits” attribute.  Register directives may have a “Namespace” or “Assembly” attribute. Examine the page and see if you can identify a problem control. 

For example, the case that led me to post this had a very specific page that was not behaving as expected.  I looked at the page in the hive, and found a suspect control on the page, named SLActivityToolbox and a Nintex.Workflows.ServerControls assembly.

Install dotPeek on the server.  It should be safe for production boxes, but if you can, do this in a test environment.  You don’t have a test environment?  You are not ready for this post then!

Examine the assemblies in dotPeek.  Generally, they will be in the GAC.  However, some 3rd party tools may be in the \bin folder of a web app, or even in the 14 hive.  In my case, I opened Nintex.Workflows.ServerControls and started reading code.  There, nestled in the ‘Render’ method, I found a suspect.  It’s a given you’ll need to know some .NET code to do this, but the times I’ve done this, it’s usually fairly easy to read and figure out at least some problem areas.

Rule #2:  Use your Enemy’s Weapons Against Them

In some cases, just perusing the code in dotPeek is enough to get to a solution.  In other cases, it takes fighting back.  Enter PowerShell, which as I’ve said is just command-line .NET.  This means that you can load .NET assemblies and tinker.  You see where I’m going. 

I imported Nintex.Workflows.ServerControls with code like this:

$x = [Assembly]::LoadWithPartialName("Nintex.Workflow.Assemblies")

This post has a good overview of loading assemblies from various sources. Loading an assembly in this way is similar to adding a reference in a .NET project.  All of the code within the assembly is now available in the commandline.  Use it wisely, grasshopper.

From reading code, I suspected a method call SnippetRepository.GetAll() was the culprit.  However, it needed a SPSite instance for me to call it.  And here is where it really clicks that PowerShell is really just .NET.  That SPSite instance can come from the built in SharePoint PowerShell commands:

$s = get-spsite http://somesite
$r = new-object Nintext.Workflow.ServerControls.SnippetRepository $s
$r.GetAll()

We get a SPSite instance and pass it to the constructor, then call GetAll.  This call threw an exception, but confirmed my suspicions. Armed with this, I was able to chase down a modified library on the site and fix it, correcting the problem.

Rule #3  Use for Good, Not Evil

Another appropriate title for this post could be “How to debug SharePoint like a crazy man.” Use these techniques at your own risk. You can seriously hose a farm doing this, so only attempt in situations where you have nothing to loose or really know what you’re doing. In my case, unfortunately, it was the first.

Since the code I ran only queried and did not update, I felt somewhat safe.  Plus, this was in a test environment, but even so, this probably wasn’t the safest debugging session I’ve ever done.  That said, you do what you have to, and in extreme cases, it’s good to know you have some heavy weapons like dotPeek and PowerShell to bring in on particularly hairy situations.

Apps You Should Have (Not Just For Nerds)

I recently posted about Trello as a service that I think even non-IT-professionals may find useful.  Continuing that vein, here are four other services that I think anybody with a computer should have.

Evernote

Evernote is a “second brain” (no comment on the first one).  It runs on just about any computer and phone, and lets you very easily capture text, photos, files, and more.  Every note gets synced to every device, and there is rich support for tagging and search.  You can quickly jot notes, send emails into it, and even snap pictures of business cards and labels you want to remember, then pull them back on any device. Here are some ways I use it:

• Notes for work.  I constantly have to remember “what did I do to fix XYZ”.  Now, if I fix something, I’ll record it in Evernote.  I have literally hundreds of solutions in my own personal “Knowledge Base”.
• Shopping lists.  For trips to “the Man Store” (ie Lowes), or grocery, I’ll jot down what I need and refer to it at the store.
• Paint numbers.  Whenever I’ve painted or had things painted, I snap a picture of the label they put on the can that has the color numbers.  They can then use this to match a color later.
• Favorite wines/beers.  If I find something I like, I snap a picture of the label on write it in a note so I can remember later.
• Gate codes.  I put in gate codes for storage and the pool.  Since notes are marked with a location, I can easily see the notes near me and pull up, say, the storage gate code when I’m at the storage location.

So, go get Evernote.  If you want to read more about how to use it, Evernote Essentials is a great book about exactly how it works and things you can do with it.

Carbonite

If you don’t have _some_ backup for your photos and other files, then you are doing it wrong.   Your computer will fail, and when it does, all those picture that only live on your computer are going away.  There are lots of ways to do backups, but Carbonite is dead simple and cheap.  For $5 month you get unlimited backup of all of the important files on your machine. 

Google Reader

News readers let you get content from just about any website or blog in a easy-to-read and relatively ad-free user interface.  Think of it like an electronic newspaper where you control what makes the front page.  In fact, when Simeon asks “what you doin on your iPad?” I reply “reading the newspaper”. Of all of the news readers I’ve tried, Google Reader is the best.  I can check dozens of sites in a matter of minutes, as opposed to wasting time browsing.  As a result, I’m always up to speed on news in my field, hobbies, as well as general national and local news.  If you have an iPad or iPhone, Reeder is a great client for Google Reader.

IFTTT

If This Then That automates tons of online services in a very user friendly way.  I know that sounds like something only nerds would care about, but here are some ways I use it, only some of which are uber-nerdy:

• When it’s going to rain tomorrow, it sends me an email.  This alone is worth it.  I don’t care about the weather unless it’s going to rain tomorrow!  Unlike those annoying weather alert sites, these are short one-line notices.
• When there is a new movie on Netflix, it emails me.
• When I post to my blog, it posts automatically to Facebook, Twitter, and LinkedIn.
• When I upload a painting to Flickr, it posts automatically to Twitter and Facebook, and uploads to DropBox.

If you have a blog, those automatic posts to FaceBook and Twitter are gold!