GregDotNet

Enterprise Web Framework talk at WNYDNUG

2008-09-12T14:08:00.004-04:00

On August 20th, I gave a talk at WNYDNUG on the Enterprise Web Framework. The slides are available here.

Western NY .NET User Group kick off

2008-02-04T23:33:00.001-05:00

On February 20th, I will be speaking at the Western NY .NET User Group. This is the first meeting in awhile, and it should be a good time. Since they haven't heard it yet, the topic will be Automatically-Generated Data Access Layers.

VDUNY Talk

2007-10-29T18:02:00.000-04:00

On Wednesday, October 24th I gave a talk on automatically-generated data access layers to the Visual Developers of Upstate New York group. You can download the slides and the sample solution here.

Microsoft Launch Event

2007-02-08T19:36:00.000-05:00

Andy Beaulieu's CNY group just held a successful (as always) launch event for the new Microsoft products. I gave a talk on the new Windows Presentation Foundation, and the slides for that talk can be found here.

CNY Developer Group Talk

2006-10-09T11:50:00.000-04:00

I recently gave a talk at the Central New York .NET Developer Group on data layers and on the benefits of code generation in general. The zip of the slides and the solution I used are available here.

If you plan to do more than just look at the existing code, I'm the first to admit that it requires some setup. Making changes and regenerating the code would require CodeSmith 2.6, SQL Server, and some configuration. In the future I plan to make instructional videos for both the use and setup of these tools. Until then, feel free to post questions.

Atlas gotchas

2006-02-13T01:48:00.000-05:00

I recently started playing with Atlas, Microsoft's framework for AJAX-like behavior in ASP.NET web applications. First of all, the framework is great, and I've already had a lot of success applying updatePanels to existing web sites.

Let me tell you, though, about the two lines of "code" that had me debugging for 14 hours.

The first problem I encountered was autopostback drop downs doing an asynchronous postback to the server, doing the work (I could see it in the debugger), and then having the results disappear into the ether instead of showing up on the screen. The problem was totally silent and caused no run-time errors of any kind (unless you count not showing the user their data as a run-time error).

This problem was caused by a nondescript piece of html that I had never explicitly written, but was rather leftover from some sort of automatic Visual Studio 2003 template html. My .aspx page had the following line:

<meta name="vs_defaultClientScript" content="JavaScript"/>

Apparently this hijacked Atlas's script handling ability and silently squelched some sort of important behavior that resulted in the above problem. Deleting this line made the problem disappear instantly.

The second problem I had was also related to autopostback drop downs. This time it was when I put an updatePanel in a dynamically added child control. The error here was more obvious. When I changed the value of any autopostback drop down, I received a client-side popup window with the following message:

"An async postback was caused by a control within UpdatePanel '' but the panel could not be found in the page."

After several hours if experimenting with dynamically-added controls (suspecting they were to blame - they were not), I discovered the real problem. Another line of code I never explicitly wrote was breaking Atlas. This time it was in the web.config file. The line was in the portion of the web.config file and looked like this:

<xhtmlconformance mode="Legacy"/>

This beauty of backwards compatibility cost me my whole weekend. Documentation for the element/value pair states that it "Reverts a number of rendering changes made for conformance to the v1.1 rendering behavior." I suspect this line was added by the Visual Studio 2005 migration wizard. Changing the value to "Strict" or "Transitional" (or by deleting the line, which defaults to Transitional) instantly fixes everything.

"The Source Control Experience"

2006-01-29T10:05:00.000-05:00

I've encountered a lot of puzzling and disappointing issues while using source control recently, and while I can't fully explain all of the problems I've encountered (seriously - there are inexplicable problems), I can share what I collectively dub "The Source Control Experience."

The fundamental problem with source control is that it isn't transparent enough. It tries to be, but obviously fails. Almost every operation developers perform on a daily basis becomes more complex in a source-controlled environment. This brings up one of my most-often-asked questions - "Does anyone actually use this?" (and the corollary - "How can it be this bad if people actually use this?").

The best recent example is migrating from a Visual Studio 2003 solution to a Visual Studio 2005 solution. As if that procedure weren't complicated enough on its own, enter source control (I can hear support representatives cringing when the customer says "by the way, it's in source control").

For my company and many others, the most common thing we want to do is to take a source-controlled Visual Studio 2003 solution (which contains a mixture of libraries, web projects, and setup projects) and send it through a process that results in a Visual Studio 2005 solution that contains the same projects, still functions (bonus), and is still under source control with history preserved.

It turns out that this exact scenario, which I still think must be the most common use case in the development world (excluding HappyExampleville, which may be a larger portion of the development world than I thought), is totally impossible.

After taking a moment to reflect on the five-odd years of development it took to arrive at the intersection of most-desired and impossible, let's explore why it's impossible. The most basic reason is that web projects are explicitly removed from source control during a conversion. Attempts to add it to source control in the same location it used to be in is a total disaster (do not do this). That's pretty much the end of the game right there. I ran into other problems (none of my setup projects converted properly) that I'm somewhat confident I could have found a way around had I actually tried. Since the game was over after the web project issue, I didn't even try.

SourceGear, the makers of what I feel is the highest-quality source control product available at the moment, attempted to come up with a procedure to convert to a solution to Visual Studio 2005 without losing history. They weren't able to.

The Microsoft-recommended procedure of "take it out of source control, copy it somewhere, convert, then re-add" should have provided warning. Warning maybe not of poor conversion capabilities, but warning that they don't think their users are interested in using source control, or, if they are using source control, they don't mind blowing away their history completely. By the way, have you ever tried to completely remove something from source control? Visual Studio suddenly becomes the Oracle ("Some time ago, this solution or one of its ancient ancestors used to have or know about source control bindings. Visual Studio does not know what to do, so it will silently remap some of your working folders to confuse you in the future.").

The fact that the community isn't in a giant uproar about this makes me wonder, again, if anyone actually uses source control in an actual environment to do actual work.

One small think I'd like to add is about the new Visual Studio 2005 source control API. This was supposed to be the silver bullet that saved source control. It was supposed to allow renaming, moving, deletion, and it was supposed to have much more coarse-grained communication to improve performance over WANs. It was supposed to fix every problem everyone ever complained about in source control support boards. Well, it didn't.

I have folders that check in perfectly using the standalone Vault client, but never succeed when using Visual Studio. When another developer deletes some files from a solution, my Visual Studio is smart enough to know they're gone, but not smart enough to know they were deleted (they show up in pending checkins as "Files to add" until I do a Get Latest). It also constants wants to check in PDB files, and then fails when it attempts to, and then decides it doesn't want to check them in anymore after I do a get latest (actually, my bin directory has two PDB files in source control, and the rest of the files aren't - consistent).

The most amusing (if gaping holes in software functionality can be called that) thing about the way Visual Studio 2005 handles source control is what it chooses to put in transactions and what it chooses to leave out. All checkins go in a "pending checkin" queue that gets committed as a transaction at some later time. Well, deletions don't go in that list, they happen immediately! This makes for fantastic broken builds, and leads me again to my question: How can anyone actually have used this for any period of time and thought that it worked well? Does anyone actually use this to do real work?

Anyone who uses any sort of automated daily build (CruiseControl, etc.) would have noticed this problem immediately. This leads me to believe that no, nobody does actually use this to do real work. Our software development practices are being driven exclusively by residents of HappyExampleville, and this is hugely disappointing.

Multi-threaded apartment saves the day

2006-01-13T01:42:00.000-05:00

You know the obligatory [STAThread] attribute that appears before every Main() method you've seen in your C# life? No, it doesn't stand for "start thread." It stands for single-threaded apartment. Recently, this caused a major problem for me.

We have several programs that read enormous amounts of information from a SQL Server database, transform the data, and export it to Excel. These programs were horribly, horribly slow. Not horribly slow in an "I'm doing a ton of work" way - enormously slow in an exponentially slower, scary kind of way. In one case, a 10% growth in database size caused the export time to rise from 1 hour to 3.5 hours.

I immediately began to suspect our code, and watching Task Manager revealed that the memory footprint of our application grew over time, and so did its CPU utilization (by a more dramatic margin). I went off in search of a memory leak. Since the program was dealing with too much data to hold in memory at one time, it was reading data in small chunks, doing its work, and throwing the chunk out (or so we hoped) before moving onto the next chunk. Similarly, the StreamWriter we were using did not hold the entire output file in memory before writing it. It flushed itself after every chunk and then appended future chunks to the same file.

The code that was reading the data was well-tested, often-used, trusted code that we've never had problems with before. I began to suspect the StreamWriter was to blame. Maybe there was some underlying buffer that wasn't getting flushed. I tried destroying and recreating a new StreamWriter for every chunk instead of having one for the entire program, hoping to achieve more linear performance (if slower). This did not work.

I began to suspect some underlying problem with our data layer - more specifically, a DBConnection object that wraps a SqlConnection object. At Red Stapler, our data layer is our most robust, most used, and more important core asset. To have a memory leak in it would be embarrassing. Still, I checked it out. I couldn't find anything.

After a bit of googling, I discovered people with a similar problem. It turns out that SqlConnection objects leak memory (more specifically, they leak handles) in certain circumstances. One of these circumstances is using them in a single-threaded apartment. The message board discussions I found offered several solutions to the problem, one of which was changing from a single-threaded apartment to a multi-threaded apartment.

Fortunately, this was as easy as changing [STAThread] to [MTAThread]. Magically, changing one character in one file, with no other changes whatsoever, fixed the whole problem and suddenly the operation that took 3.5 hours to perform now took 20 minutes. The memory footprint was very stable and the CPU utilization did not increase throughout the program.

I was relieved to find there was nothing wrong with our logic, and even more relieved to find there was nothing wrong with our libraries. The only problem was the lack of understanding regarding STAThread and MTAThread (the latter of which I didn't know existed until solving this problem). Documentation in Visual Studio was poor, only mentioning that these attributes only matter when using COM. This reveals two things: SqlConnection must use COM under the hood, and the only thing I need to know about this is that MTAThread magically solves this horrible problem.

Later, I did do some more research on multi-threaded versus single-threaded apartments, and from what I gather, single-threaded apartments have some sort of built-in thread serialization
mechanism that protects the developer from his or herself. In the future, I will probably use MTAThread unconditionally - certainly when using SqlConnections.

ASP.NET 2.0 Profiles - The good and the missing

2006-01-03T01:05:00.000-05:00

I recently started using the Profiles feature of ASP.NET 2.0 to store user preferences (search filters, etc.) for a web app. Profiles make it very easy to store and retrieve per-user data without a lot of configuration. They are strongly-typed and way better than using session state or rolling my own database solution.

Immediately after I noticed how great Profiles are, I noticed how obnoxious it was to write the C# code to store and retrieve the profile data. This was the perfect opportunity for me to use a feature I've never used before: Data binding.

I open up the designer for the web control I am working on and I select one of the text boxes that I use to filter a search. In the properties pane, I go to the Data Binding section only to find that binding to a Profile value is not an option. Since I never use data binding, I assumed I just didn't know how to do it properly and began researching. Nowhere did I find anything about binding to Properties. Very disappointing.

What I did find, however, was almost as disappointing. I found huge articles on how to bind data grids and other controls directly to your database. Searching the articles for "validation" returned no results. It seems a bunch of people are fond of allowing users to have a validation-free window directly into their database. I'll refrain from ranting on this particular topic right now, but I wouldn't be surprised if it comes up again in the future.

It's unfortunate that data binding is unavailable in the only instance I'd ever want to use it, but Profiles are still better than session state.

The rocky transition to 64-bit

2005-12-23T09:56:00.000-05:00

Red Stapler Software recently purchased a new 64-bit server, and I began the exciting endeavor of installing Windows Server 2003 x64 Edition for the first time. I'm very happy with the outcome but there were a lot of hiccups along the way.

First, the decision to buy 64-bit hardware was a very easy one. It's almost impossible to get 32-bit server hardware nowadays, and for good reason. The x64 architecture is faster and fully backwards compatible. No brainer. The same can't be said for the operating system and software.

Microsoft support for 64-bit is almost perfect. The huge gaping hole in their support of it is the lack of any 64-bit edition of Windows Server 2003 Small Business Edition. Fortunately, I was interested in installing Windows Server 2003 Standard and SQL Server 2005, both of which have x64 editions (and Itanium, for those who are interested).

While installing the operating system (just after discovering that my USB keyboard worked in the BIOS - impressive), the only problem I ran into was acquiring 64-bit RAID drivers. Actually, they were easy to acquire, but since Windows had no native support for my RAID controller at all, I had to download them from the internet and put them on a FLOPPY DISK. Windows Setup will not explore the wonders of compact discs when looking for drivers. Since the server we ordered had no floppy drive, I had to dangle one outside the case until Windows was installed and happy with its new driver.

Since we write all of our web sites, tools, and other software in .NET (1.1 and 2.0), 64-bit compatibility for our software was a breeze. It isn't even necessary to recompile. All of our 1.1 applications run fine on the 2.0 framework, but we do have the option to have .NET 1.1 installed side by side and run older applications on that platform instead. Things become slightly more complex when running ASP.NET web applications in IIS.

IIS is capable of hosting ASP.NET web applications in a 32-bit or 64-bit process (but not both). There is also the option to host on the 1.1 or the 2.0 platform (or both), but since we had no compatibility issues, we chose to use 2.0.

At first, I tried hosting all of our web sites on ASP 2.0 64-bit. This worked for almost every web site. The web site that did not work referenced two third-party utilities: Aspose.Word v2.5 and ActivePdf Toolkit v3.5.2 SP6. It turns out that both of these utilities use COM, and making an in-process reference to a 32-bit COM component in a 64-bit process is a no-no. This was a deal-breaker.

My first response was setting IIS up to host the web sites in ASP 2.0 32-bit mode. This worked. The problem web site and its associated Word and PDF automation tools worked perfectly. Unfortunately, it broke SQL Server 2005 Reporting Services. Reporting Services for the 64-bit version of SQL Server 2005 uses IIS and requires that it be hosted in ASP 2.0 64-bit mode. Since it's impossible to host it in 64-bit mode while also hosting our problem web site in 32-bit mode, we were stuck.

I turned my attention to the tools. I contacted Aspose's support (which was fantastic - I received an answer over instant messager immediately) and was told to upgrade (free) to version 3.x of their product. This fixed the problem immediately and required no code changes. One down, one to go.

I contacted ActivePdf's support, and during the week it took them to get back to me I tried downloading the new version of their Toolkit (4.0). After modifying the source code to conform to the new API (which had undergone meaningful changes like BinaryImage becoming a method instead of a property, forcing me to add parenthesis), I encountered the same problem as I did with our existing version. Eventually they admitted that they don't officially support 64-bit platforms and they had no solution for me.

Since I had such a great experience with Aspose, I downloaded a trial of their Pdf form filling tool (Aspose.Pdf.Kit). I had it up and running in 10 minutes, and had our code totally converted in half an hour (and it was much cleaner). This tool worked in our 64-bit environment without flaw just as Aspose.Word did, and all of our 64-bit migration barriers were gone.

Standalone 32-bit applications such as WinZip run without incident on a 64-bit server (note that 32-bit applications get installed, by default, in C:\Program Files (x86)\ rather than C:\Program Files). In-process COM is what you have to watch out for.

The switch to 64-bit was well worth making for us, but the more third-party (well, non-Microsoft, really) tools you reference from web applications, the harder it will be. Of course, the biggest winner in the move to 64-bit is SQL Server, so a very reasonable option is to have a 64-bit server that serves just the database and have the web server run a 32-bit operating system.

Passing arguments to ClickOnce applications

2005-12-18T13:31:00.000-05:00

My company, Red Stapler Software, has a tool we call the Transfer Utility that we use to deploy and update our web applications. Since our live server, our testing server, and each development machine uses this tool, an upgrade to the Transfer Utility results in a lot of work by a lot of people on a lot of machines.

To combat this, we decided to make the Transfer Utility a smart client deployed using ClickOnce. Now when we update the tool, we simply publish to our live server, and every machine that runs the the tool (including the live server it was published to) will automatically pickup the latest version. This is great, but there were a bunch of problems along the way.

First, there are two types of ClickOnce applications: Ones that can run offline and those that cannot. We chose the type that can be run offline so that developers can still work if their link to the live server is interrupted for whatever reason. For this type of ClickOnce application, there are two ways to run it. The first is from the Start menu shortcut (which is not a true shortcut in the sense that if you right-click on it, it does not bring up the normal options such as the path to the file it will be running) and the second is by visiting the URL to which the ClickOnce application was published.

It turns out that the Start menu shortcut method of invocation has a number of drawbacks. First, I was unable to find a way to make upgrades to new versions of the Transfer Utility completely transpartent. It always prompts the user. This is a problem when invoking the utility from the command line or from a scheduled Windows task.

The URL method of invoking the application can be made to upgrade and run transparently, with no prompting at all. To do this, simply create a desktop shorcut of the URL and add the domain to which your ClickOnce app is published to your Trusted Sites list. After doing this, running the desktop web shorcut (even from the command line) will run the program without asking you for permission and without prompting you for upgrades.

This is great until you actually try to run a real application that lives outside of Happy Exampleville, such as those with command line arguments. This brings us to our second problem with the Start menu method of invoking ClickOnce applications: I've been unable to find any way to pass command line arguments to them.

Once again, I've had more luck with the URL method of invocation. To pass command line arguments to your ClickOnce application, simply append "?commandLineArg1[&&commandLineArgN]" to your URL. This works nicely, except that inside your program's Main( string[] args ) method, you'll find your args array very empty. In order to extract command line arguments, you'll have to do the following:

string fullUrl = AppDomain.CurrentDomain.SetupInformation.ActivationArguments.ActivationData[ 0 ];

This will give you the full URL of your application. You'll need to split the string by the question mark and parse the arguments that appear after it. Remember to HtmlDecode (in System.Web.HttpUtility) the string before you parse it. Oh, and you'll also have to check AppDomain.CurrentDomain.SetupInformation.ActivationArguments.ActivationData for null before you do this, because if someone does try to invoke the program using the Start menu shortcut, there won't be any activation data and the line of code above will throw a null pointer.

We now have the ability to run our application in one click with no prompting even while passing command line arguments to it. The remaining problem is that we don't want to have to create one web shortcut for every combination of command line arguments we ever want to pass to our program. To accomplish this, I actually had to create my own batch file, which looks like this:

"C:\Program Files\Internet Explorer\IEXPLORE.EXE" http://files.redstapler.biz/transferutility/redstapler.transferutility.application?%1

I called my batch file executeTransferUtility.bat. Note that I'm only passing one argument. You can choose to add more (%2, etc.) or you can choose to mash them all into one with some delimiter (like '/') and parse them yourself. Now I can finally use my ClickOnce application like a normal program from the command line: 'executeTransferUtility auto' will invoke the Transfer Utility with one command line argument, "auto," which incidentally makes the utility perform its normal set of batch operations associated with the machine it is installed on without asking the user any more questions.

That seemed unnecessarily exhausting, but it solved my problem. Since I'm forcing the use of the URL method of invocation, I probably won't be able to run the application while disconnected. I also suspect that there exist (hopefully) easier ways to do what I've done, or that there is a way to make Start menu method of invocation accept command line arguments. Still, I've failed to find any good documentation to walk me through what I'm trying to do, and I suspect I've strayed too far from Microsoft's expected usage pattern for ClickOnce and I'm on my own. I look forward to reading about more solutions to the above problems.