Beware CORRIDOR MRO software

I want to warn people today about a specific piece of software known as CORRIDOR. Produced by Continuum Applied Technology, this program is a mess of poor design decisions that I unfortunately have to deal with. I'll set aside what I consider flaws that are really more matters of taste, such as using Oracle as the back-end database (I could spend years ranting about how Oracle and PL/SQL need to be set on fire; use DB2, PostgreSQL, or SQL Server instead) and thinking that putting the word "Please" in front of a UI resource string somehow makes the system user-friendly. I'll also remain mum on the pricing model, save to say that you'll be paying Continuum forever for practically anything you need if you actually buy the software. You may actually find SAP Business One is cheaper (!!!) over time. Instead, I'm going to focus on the major flaws in the product's architecture.

First, CORRIDOR is constructed as a C++ user interface layer over what appears to be a business object layer (called "Rita," at least to those of using the API). These two components form the UI, which connects and writes directly to the Oracle database using the full version of a specific Oracle client. This two-tier architecture is a major problem for maintenance on the end user and ISV side, as we can't run different client versions against different server versions. We can't upgrade the Oracle client without possibly breaking its ABI to the Rita layer. We can't upgrade Rita without possibly breaking its ABI to the actual UI program. Perhaps most importantly, we can't upgrade the database at all -- whether in schema or version -- without possibly breaking everything. So, if I want to upgrade my version of Oracle to fix a security flaw, I better hope it doesn't need a new client, because then I'm stuck. Likewise, I can't run multiple different minor versions of the client software if there were any database changes, because then the logical interface between Rita and the database will no longer function. Finally, this also means that any applications that use the poorly-documented API (more on that later) have to have:

• A full installation of the CORRIDOR software, both UI and Rita
• A full installation of the Oracle client
• Some extra undocumented dependencies in the CORRIDOR directory if they're a .NET application

This is one of the reasons why the three-tier model, where clients talk to application servers and application servers talk to databases, all through fixed interfaces, is popular. If CORRIDOR were a proper three-tier application, API programs wouldn't need the entire copy of CORRIDOR to work. They'd just need the DLL that defines the interface and a pointer to the server. Additionally, if the interface is properly versioned, you'd be able to run different client versions and, at worst, get a message that your client is no longer supported.

Another major issue with CORRIDOR is the API. Any time you want to automate some process, you need to use the API, which is available in C++, Java, and .NET CLR versions. The C++ API is essentially a set of calls directly into the Rita DLL, while the Java and .NET versions are wrapper classes atop this DLL. Unfortunately, the .NET wrapper is very lightly wrapped. Oracle exceptions are manifested as native memory corruption exceptions, regardless of their cause. Major issues manifest as requested (!!!) abnormal program terminations in the C++ runtime used by the Rita DLL, which result in your program unpredictably and irrevocably crashing. Communication is handled by passing Hashtables with const strings to identify what you want to do.

All of that would be fine to suffer through except for the fact that the API isn't really documented. At all. Oh, sure, there's a help file you get that explains what parameters are used by each "request type," but key information is missing. For example, if the Oracle connection fails, you get an exception indicating that native memory is corrupted. You don't get any indication as to why, exactly, the Oracle connection failed. One common cause, as I discovered, is that you need to encrypt the password before passing it in, using an undocumented function. Other critical missing information is the format and type of the primary key to select items in the database (it's claimed to be the "lot number," but the API's notion of a lot number is different from the UI's, and it's not a unique key in any event) and the meaning of certain terms (e.g., "system status" with regards to an item and "lot extension numbers"). Some of the information is just patently incorrect, such as the required parameters for certain API calls.

Normally, none of this would really annoy me, because I'd just skip past the program and go right into the database. However, indications thus far are that such access isn't allowed -- not that it's unsupported, but that you can't get the username and password for direct table access even if you want it. While there are definitely ways around that, this sort of disregard for the needs of users and ISVs is reprehensible. If I purchase your software outside of a SaaS agreement, it's mine to do with as I please. If I have to buy the database software, you better believe I have the right to access the database as I see fit.

I said I wasn't going to rant about the cost of CORRIDOR, but I'm going to break that promise and point out that you have to pay for every API that you want to use and for which you want support. That means that, normally, you can't use the API for anything. You have to pay for that access. In fact, almost everything you want to do that isn't "use the UI" is a pay feature. It's nice to know that Continuum is taking a cue from Oracle in selling crippled software, then charging for upgrades to the full version. I expect full API and database access from ERP software. SAP's been doing that for years. It's a standard feature at this point, not a pricey value-add.

So, to sum up, I really dislike CORRIDOR, and I don't feel it's appropriate for production environment use. Some of my associates are looking at other programs, such as Pentagon and Quantum, and I'm sure I'll have some commentary on them as well. I would say that, for now, you should avoid CORRIDOR -- and in general, you should always remember that ERP software packages are like sewers. Some of them are nicer than others, but they all still stink.

DPM 2010 RC and the MSDEWriter for SQL Server 2000 on a mixed platform

Lately I've been playing around with the DPM 2010 release candidate. For those who don't know, Data Protection Manager (DPM) 2010 is Microsoft's latest enterprise backup software. One of the neat features it offers over junk such as Asigra (offered by a variety of third party service providers, including one local to the Memphis area called Electronic Vaulting Services) and more useful software such as Backup Exec is its deep connection into Windows -- specifically, the wonderful Volume Shadow Copy (VSS) service. Shadow Copy allows you to make real-time snapshots of data, which is quite handy when backing up systems such as SQL Server and Exchange. Both of these have backup functionality built-in, but these "backups" -- data dumps, really -- have to be done first, then those files have to be backed up. This can create a long delay between the snapshot time and the actual backup time, exposing you to risk. Also, without VSS, you can't do any real-time copying of changes. In a 24/7 operation such as the one I support, we can't afford an entire day of exposure. We'd like to have zero minutes of exposure (hot standby), but that's for another day.

Generally, DPM 2010 is pretty slick. It automatically detects tape libraries and intelligently manages them, letting you know via alerts in its console or in System Center Operations Manager when a tape needs to be swapped out. You can specify short-term and long-term retention goals, which means you can hold data on disk, then periodically do full backups to tape. The real downsides to the RC are that the disk backups require full ownership of an entire disk or array (partitions aren't sufficient), USB and 1394 devices aren't supported, and some of the functionality doesn't work -- such as the AD schema extensions for self-service restores by users. Also, it's an agent-based backup solution, so it won't work on any systems that aren't Windows-based. You can work around this by writing batch files or PS scripts to copy snapshots to a Windows server for backup, but that's clunky and suffers from the time delay issue I noted earlier.

One fun feature of DPM is that it has both 32-bit and 64-bit agents. It selects the agent based on the architecture of the target OS. I say this feature is "fun" because it can cause some issues with mixed-architecture software. If, for instance, you run 32-bit SQL Server 2000 on a 64-bit platform, the DPM agent can't see the 32-bit instances because it talks to the 64-bit VSS service. The 64-bit service only looks for 64-bit sources, so your instances don't show up in DPM. The way to get around this if everything else is working (i.e., when you use vssadmin list writers from a command prompt and you see the MSDEWriter, which is the SQL Server 2000 VSS "driver") is to make a registry edit.

In the registry, go to HKEY_LOCAL_MACHINE\Software\Microsoft\Microsoft SQL Server. Create a new key here with the name of your instance. Then, create a new key under that key named "MSSQLServer". Then, create a new key under that key named "CurrentVersion". At this point, you should be at HKEY_LOCAL_MACHINE\Software\Microsoft\Microsoft SQL Server\\MSSQLServer\CurrentVersion. Create a new string value named "CurrentVersion" here and enter the string "8.00.194". The astute will note that this matches the same structure you can find under the Wow6432Node -- basically, we're copying enough of the structure to let the 64-bit VSS system "see" the instance.

Now, next time you connect with DPM to initiate protection, you should see the instance you set up in the registry. Props go to Hitesh Sharma of Microsoft for this (http://www.eggheadcafe.com/software/aspnet/32831431/data-protection-manager-n.aspx is the original thread where the solution became apparent).

Beware of sysaudio.sys!

There's a new Google hijacker running around that can install even on fully updated copies of Firefox 2. The hijacker in question drops a file named sysaudio.sys in the %SystemRoot%\system32 directory (not to be confused with the one in %SystemRoot%\system32\drivers). When this file is loaded, it silently hooks the ws2_32.dll calls made by web browsers and watches for requests to search engines. When it sees them, it pulls fake search results through script injection from http://1.2.3.0, which is a trampoline to another site. The version that infected my SO's computer happened to go to 94.247.2.58 (hs.2-58.zlkon.lv -- *.zlkon.lv being a known purveyor of malware and spam). The file doesn't appear to do anything else, short of being a little interesting if you open it in notepad (some of its imports are repeated backwards).

The big threat here isn't the search results, which are basically useless and an annoyance, but rather the fact that the owner of that site could easily alter it to deliver much more devastating software. There's also the little problem that there's not a lot of information on this running around and that major AV packages (F-Secure and NOD32) don't see anything wrong with the file. Malwarebyte's Anti-Malware detects it as "Rootkit.Agent," though it doesn't seem to have any rootkit properties (no kernel hooks, for instance, at least not detectable by Rootkit Unhooker).

So, watch out for this file, and consider blocking the above address at your router. As I noted, it can drive-by install on a fully patched Firefox 2. Whether DEP can stop it, I don't know; the machine I found it on only supports software DEP, and it was in opt-in mode.

UPDATE: Firefox 3 is vulnerable to this as well. I don't believe Chrome is, however.

OnDeserializationAttribute, Dictionary members, and GetHashCode

As it turns out, the Dictionary<T, U> generic collection in the .NET Framework uses custom serialization and builds itself in its OnDeserialized method. What this means is that, if you deserialize such a Dictionary (using any of the Formatters), it will proceed to deserialize its keys and values and add them back to itself. No problems there, right?

What's a bit surprising is that the keys and values do not have any of their custom serialized members (such as, oh, their own generic Dictionary members) deserialized before that happens. This could prove a real problem for you if you override object.Equals, implement IEquatable<T>, and override GetHashCode. When a key is added to a Dictionary, it is added by hashcode (no surprise). That hashcode is then immutable, at least as far as the Dictionary is concerned. Changes to the key that alter the hashcode do not change the hashcode for the object that the Dictionary remembers. Normally, that's not a problem, but some methods, such as Dictionary<T, U>.ContainsKey, manage their fast operation by comparing hashcodes -- in the case of that method, it looks for a key with the right hashcode, then calls key.Equals(candidate). These methods, therefore, will not perform as expected if modifications are made to the keys that affect their hashcode. Optimally, the keys should be immutable.

All discussion of the innards of the Dictionary class aside, the real point here is that if you have keys in a Dictionary that are mutable objects (let's call instances of class Entity) whose hashcodes can change based on the values of their members, and any of those members are themselves Dictionaries, the hashcodes that the Dictionary<Entity, U> object stores will be incorrect on deserialization. At the time the Dictionary of Entities is deserialized, the Dictionary members of the Entity instance have not yet been deserialized, so the Entity object's hashcode is incorrect given its actual data. You'll be in the fun position of having keys that are in your Dictionary (via the Keys property) appear as not being in your Dictionary (according to ContainsKey). This can be a real pain to debug, especially if you're working in ASP.NET and you have relatively limited visibility into the automated (de)serialization process.

In the case above, the solution is to add an OnDeserialization method to your Entity class and mark it with the OnDeserialization attribute. In that method, you should call the OnDeserialization method for each of your dictionaries. Currently, the implementation of the Framework is such that your OnDeserialization method will be called before the Dictionary calls GetHashCode on the instance (thankfully, as it wouldn't make sense any other way), and your hashcodes will be correct.

(See http://connect.microsoft.com/VisualStudio/feedback/ViewFeedback.aspx?FeedbackID=322952 for Microsoft's confirmation of this behavior and solution.)

Don't forget to surface BAPI_TRANSACTION_*!

When working with SAP and the BizTalk Adapter Pack, you've got a lot of opportunities to make mistakes -- especially when working in RFC mode, the mode that's recommended by Microsoft (and, to a certain extent, me, as it makes life easier having all your BAPIs on one object and being able to combine multiple BAPIs into one transaction). One newbie mistake is forgetting to surface BAPI_TRANSACTION_COMMIT and BAPI_TRANSACTION_ROLLBACK. Don't forget these guys! Many, if not most, BAPI functions in modern versions of SAP don't auto-commit, and you'll be left wondering why SAP said your changes were saved and documents were created but nothing shows up in your installation if you don't remember to surface and call these BAPIs as necessary. "As necessary" means checking the Messages in the returned BAPIRET2 table and looking for messages with no TYPE or a TYPE of "S" (for Success, naturally), then calling the rollback BAPI if you find some messages that don't fall into that category. Otherwise, you should call the commit BAPI.*

Oh, you're probably wondering what BAPIs and RFC mode are. Those will have to wait for another day.

* Note that SAP can generate some warnings and still succeed in performing the task. This is common when working with Sales Orders if two Customer PO Numbers (BAPISDHD1.PURCH_NO_C) are the same. Also, SAP can fail to perform a task without generating any error message; this is common when working with documents associated with the Business Object, such as SAPTexts or Order Schedules. If one of those fails to create (because of incorrect parameters or insufficient information), you won't receive any message about it at all. Otherwise, you'll receive a Success message. I'll have more on SAP's "error information" -- and those are massive sarcasm quotes -- at a later date.