Solution to SAS Errors in StatTag – Analysis and Resolution

As noted in this GitHub issue, we had several users experiencing errors when trying to run SAS code. Fortunately we had several diligent folks including Carla L. Warneke, MS and Ann Kan (Northwestern) who helped us get to the bottom of the issue. This is fixed as of v6.0.4 of StatTag, and an analysis of the error and investigation is below.

The Error

[16] ERROR Could not load type 'SAS.LanguageServiceCarriageControl' from assembly 'StatTag.SAS, Version=6.0.3.2, Culture=neutral, PublicKeyToken=null'.
Stack trace: at SAS.ILanguageService.FlushLogLines(Int32 NumLinesRequested, Array& CarriageControls, Array& LineTypes, Array& LogLines)

StatTag has been making use of the SASHarness project developed by Chris Hemedinger for our connection and interaction with SAS. Embedded in that code, which also came up in some Stack Overflow posts (including this one) was the need to explicitly instantiate some dummy objects to get things to load/initialize correctly from SAS.

//For some reason, these two declarations need to be here
SAS.LanguageServiceCarriageControl CarriageControl = new SAS.LanguageServiceCarriageControl();
SAS.LanguageServiceLineType LineType = new SAS.LanguageServiceLineType();

Adding those two lines should fix the exact error users were reporting. What’s odd is those two lines of code have been in StatTag since we rolled out support for SAS. It was surprising to me then that this error showed back up.

Investigation

My machine with SAS is my development machine for StatTag, so I loaded up Visual Studio and started debugging. Unfortunately, I was unable to reproduce the error locally. This is where I’m indebted to those users who gave a lot of time to work with me on testing different configurations and possible solutions.

First, some users didn’t have SAS Enterprise Guide installed. Given that I did on my machine, my first suspicion was that there was some dependency missing. I didn’t want to ask users to install EG if they didn’t need it, so I confirmed that the dependencies/references we’re using in StatTag (SAS, SASIOMCommon, SASObjectManager) are provided if you install SAS Integration Technologies. It may be that these were already installed otherwise with their SAS install. Regardless, once they tried installing, it did not fix the issue.

Because StatTag is loaded through Word and there are other dependencies that are thrown in the chain, I wanted to remove some of those. I created a SASCheck utility that would run through some diagnostics. It checks registry keys, and the first version checked to see if I can load different classes. Everything ran fine, no errors reported. So clearly all of the types were there and could be found – including LanguageServiceCarriageControl. I revised the program to then attempt a connection (I should have done this in the first place!) so that I could see if there was a failure. While I was adding that code, Visual Studio warned me about those two lines I posted above that are needed to get SAS to work. It was complaining that the dummy variables aren’t used (makes sense). And that’s when it hit me.

We know that compilers work hard to optimize our code. So what if the compiler was optimizing away those two magical lines of code because it realized they weren’t being used? I went ahead and “made use” of the variables (not really, but to the point that the compiler kept it in). So when a user tested the SASCheck utility, they reported it connected to SAS without any error.

This was good and bad. I didn’t have a baseline that matched what StatTag was doing, because I had played around with those dummy variables. But it certainly gave me a good lead. So I added similar code into StatTag that made use of those dummy variables, just like in the test utility. We went ahead with a beta release of StatTag, and imagine my delight when a user reported that it was now working!

Analysis

The root cause here was compiler optimization. Visual Studio was optimizing away those dummy variables, and so because they weren’t being loaded, the SAS types weren’t loaded and failed. So when StatTag was being compiled and released, the compiler was essentially crossing out these lines:

//For some reason, these two declarations need to be here
SAS.LanguageServiceCarriageControl CarriageControl = new SAS.LanguageServiceCarriageControl();
SAS.LanguageServiceLineType LineType = new SAS.LanguageServiceLineType();

There were a few clues along the way I missed, that are worth noting:

First – I realized at the end that I was only ever testing a Debug build of StatTag with SAS. When I was running things locally, it was always against a Debug build, never Release. The compiler optimization only happens in a Release build, so the whole time I thought I was reproducing the problem, I wasn’t.

Second – why did older versions of StatTag work? Over time, the compiler changed (I upgraded Visual Studio), and so this particular optimization showed up after SAS was first made available to users, and things were working.

Third – what was different between the old versions? I had a working version (v5.*) of StatTag, and one that didn’t work. Unfortunately it wasn’t until the end of the investigation that I thought to load them in the IL Disassembler and compare. What I discovered was that I could clearly see the LanguageServiceCarriageControl referenced in the working version, but not in the other.

Overall this was a great reminder for how I manage debugging and troubleshooting, and ensuring when trying to replicate an error that I use the exact environment (including build type) that users are using.