Migration from ASTA to Data Abstract (Delphi)

It is assumed that you have used and already understand ASTA components. Few assumptions are made about knowledge of the Remoting SDK or Data Abstract (RO/DA), but we recommend that you read the articles referenced to get a good understanding before proceeding.

Moving from ASTA to RO is not a simple step. Should I take it?

Before embarking on the process of moving from ASTA to RO, consider the benefits first. The process, whilst not technically complex, is an involved one and will require some dedicated work from you. The larger the project, the more effort is required. So ask yourself: "What am I getting out of the process? Is it worth it?"

For me, there was no doubt. I really needed the ability to produce a standalone application that used a DLL instead of a separate server. I also wanted to use an embedded database and run the application on a single PC. RO provides all this and ASTA does not.

To further make the case in favor of RO, there were other benefits that RO/DA (the Remoting SDK & Data Abstract) gave me including easier database independence and more elegant coding mechanisms.

Keep Copious Notes

I strongly encourage you to keep notes as you progress. If you are anything like me, you will probably try and squeeze the process around other, more important, work. That being the case, it is even more vital to keep notes to save you from having to "read into" the process again. Having to "read into" the process can be so demanding that it can prevent you from doing the upgrade in the first place and that is not a situation we need. Joel Spolsky (of 'Joel on Software' fame) wrote a great article on this. Read it here, especially Step 8: The Joel Test: 12 Steps to Better Code

I've included some template forms in this document for you to use to remind you of the work required to move from ASTA datasets to RemObjects datasets. I found these invaluable as they helped ensure that I didn't forget anything. It is easy to miss a small item when you're doing such intensive work and again, keeping notes will help you.

Keep a Backup Copy of your Original ASTA Application

I know that I'm probably teaching grandma how to suck eggs, but I cannot state this strongly enough. Keep a backup copy of the original code that will remain untouched as you proceed through the process. I guarantee that you'll go back to look at it now and again to remind yourself about the state of your original code.

Make a List

Before starting the process, make a list of all the work that needs to be done. Be fastidious about maintaining this list and, more importantly, about crossing an item through when you're done. You may surprise yourself about how much of a benefit this becomes. You could well find yourself doing just a bit more work so that you can cross another item off the list.

But maybe I'm jumping the gun. What needs to be on this list and how do I find it? Remember that you'll need to examine both the server and the client applications to see where ASTA has been used. Here's my advice for starting with:

• Datasets. Make a list of all the AstaClientDataset components your application uses. Each one of these will need to be updated to an RO dataset.

• Asta Business Objects. Your server application may make use of Asta Business Objects. If it does, make a list of them as each of these will need to be converted to RO. Remember that these are also likely to be accessed via AstaClientDatasets in your client application, but these were probably picked up above.

• Global Exception Handler. Your ASTA server application is very likely to make use of a global exception handler. Its default behavior is to simply handle exceptions in such a way that they do not stop the server from running and this default behavior is satisfactorily handled internally by RO. However, if you are doing any other form of processing in the global exception handler (I was writing to an error log, for example), you'll need to re-provide this in your RO app.

• ServerSocketCodedParamList. ASTA applications often make use of the AstaSocketCodedParamList event and message transport methodology. If yours does, you'll need to examine them with a view to upgrading them to use RO methods.

• ServerSocketClientAuthenticate. ASTA applications often make use of the ServerSocketClientAuthenticate method as part of login security. This methodology is completely supplanted in RO and a review of the contents of the Authentication and application security article will help.

• ServerSocketAstaClientUpgradeDecide. ASTA applications may make use of this event to decide if a client application needs to be updated. The RemObjects Hydra product provides an AutoUpdate feature - see the Hydra Autoupdate Explained (Delphi) article for details.

• ServerSocketCodedMessage. Many ASTA applications make use of the low-level messaging facilities provided at the socket layer. These would be replaced in almost all instances by RO methods.

This list is not exhaustive but it's a great starting point. You may find other items in your ASTA application that need to be moved, some of which will only become apparent after you have started the process. Should this happen, add the items to the list.

Focus

As those of us that have done refactoring know, there's a concept of 'two-hats'. A refactoring hat and a 'new work' hat. We think about doing one process or the other but we never do both at the same time. So it is with the process.

While you're performing the process, I strongly urge you to think only about the process. As you examine your code and look to upgrade it, you will undoubtedly come across circumstances where your code could do with some improvement. A little tweak here, a little edit there. DON'T DO IT! It’s very important to keep focused on the task at hand. I tend to keep a separate page to the side of my desk which I annotate with lists of code that I want to revisit. Sometimes, I also add a to-do item for ease of locating the area later. But I stay focused on the subject matter at hand.

Getting Started. Part One

This is probably the hardest part. When you start the process, you want to be kind to yourself. Give yourself an easy task and cross it off the list. Unfortunately, it rarely works out that way.

No matter where you want to start in the process, you'll have to convert the server first and the first thing that needs to be worked on is database access. Follow the instructions in the Easy Steps to Create a Data Abstract Application (Delphi) article to create a new RO server. I suggest that you do move the channel and message components to the Data Module as described in the article. You'll add functionality to this from your old server, upgrading it as you work through the process.

When defining the schema, include the declaration of database connections. (Defining the schema will require a thorough examination of your ASTA DataModule and may best be handled piece by piece rather than trying to do it all up-front).

For an RO client that hooks to that server, you need a copy of your current client application and make it 'RO/DA Aware'. Do this by adding a Data Abstract ClientDataModule via the Delphi main menu: File | New | Other … and then select the Remoting SDK for Delphi tab and the Data Abstract ClientDataModule item.

Getting Started. Part Two

So now we have a new server that contains RO/DA components and a hybrid client that contains both ASTA components and RO/DA components. As we progress we'll build more and more RO/DA elements in and remove ever more ASTA dependencies. The final target is a fully equipped RO/DA app with no ASTA dependencies at all.

As you go through the process, you may find that some things just do not transfer easily. For example, the AstaParamList has no direct equivalent in RO. Try not to get bogged down in these areas. Where necessary, make a separate note to revisit a section of code at a later time and progress smoothly through the process.

In all likelihood, having created a successful database connection in the prior step, your next task will be to manage the login process. I suggest you do not try to duplicate what you did in ASTA but follow the directions given in the Authentication and application security article instead. There is also an RO/DA sample app that shows how to perform secure logins.

You should now have a working database connection and a means of applying security to attempted logins. If you have not already done so, I suggest that you use the RO Service Tester to exercise elements of your RO/DA work as you add them (article Using the Service Tester refers). It will give you confidence in the work that you are doing and quickly highlight any problems.

Getting Started. Part Three

With a working database connection and working security, you are now ready to start on your list that you created before (see Make a List part). Again, I recommend starting with something simple like the SendCodedMessage sections. Replacing these is a simple matter of creating remote methods at the server and changing the calls to these methods in the client. Details on writing these remote methods are contained in a number of articles and are at the heart of how RO works. I recommend reading How to Write a Remoting SDK Server (Delphi) and How to Write a Remoting SDK Client (Delphi) as starting points.

Should your remote methods make use of the database, you'll need to become accustomed to creating and using commands via the Data Abstract Schema Modeler. This ensures that you get thread safe database access.

Another thing to consider here is that ASTA has the capability to perform server push. Because RO relies, in the main, on HTTP transport this is not really a viable option. You could use the Multicast event technology, as shown in the Multicast Events article, but be aware that this is dependent on the client regularly scanning the server. This act can place a heavy load on the server when there are multiple clients connected. In my own case, I changed the way my application worked to get around the situation where I had used server push.

Upgrading ASTA Business Objects

In ASTA, you can use Business Objects to perform actions similar to both RO's remote methods and Data Abstract's remote datasets.

Where you have constructed an SQL WHERE clause on the fly, note that you'll need to do this at the server now instead of the client. Pass parameters to your RO/DA method and use those to help you construct the SQL.

The best way to do this is by using the Where property of an IDADataset. However, do note that if a Where clause already exists in your SQL statement it can cause errors. It’s best to leave the Where clause blank in the SQL statement and then construct it from scratch using the Where property.

Upgrading ASTA Client Datasets to RO Client Datasets

In all likelihood, this is the section that will get used the most as you undergo the process. It’s also the longest, so I've broken down the steps into Annexes and Appendices at the end of this document. See the Appendix.

Hints, Tips & Tricks

As you work through the process, you're likely to come across some gotchas. Here's a list of the ones I came across and how I worked around them:

• CheckBrowseMode. My ASTA application made much use of the CheckBrowseMode method and there is nothing similar for the RO Dataset. Instead, I wrote a method called CheckBrowseMode in my DataModule that accepted an RO Dataset as a parameter and then did the following:

if CDS.Editing then CDS.Post;

I then replaced all of my CheckBrowseMode method calls with calls to the above.

• QuickReports (1). I used QR when creating my ASTA application and it depends upon the existence of a TDataset derived component for its data source. The RO Dataset however, derives from TDADataTable so that makes it unsuitable as a direct replacement. However, it does embed a TDataset based component in its Dataset property and that can be used for QR instead. So, for the Dataset property of your QR report and data bound controls, use 'CDS.Dataset' where CDS is the name of your RO Dataset component.

• Indexes. In-memory indexes can be built for ASTA Client DataSets, but there is nothing comparable in RO. I used the Sort method of the RO Dataset instead.

• Field Events. ASTA ClientDataSets have the ability to use the standard TDataset OnGetText and OnSetText events. Sadly, the version of Data Abstract available to me when I did the conversion had not surfaced these events. Instead, I added some code to the form's OnCreate event to hook the original methods directly to the underlying events like this:

DataTable.Dataset.Fields[x].OnGetText

• Persisting Changes. Sometimes I found that no matter what I did, edits I made to data were not persisted at the database. Sometimes I'd get an error and sometimes not but, in all circumstances, the resolution was the same: visit the schema at the server and ensure that you've set the TargetTable property and InPrimaryKey property for the relevant fields.

• Duplicate Names. Believe it or not, it’s quite possible to enter a duplicate name for a Dataset in the RO/DA Schema Builder. Needless to say, this can cause confusion as well as some very strange bugs. I strongly recommend that you use unique names.

• OnAfterPopulate. ASTA datasets have an event called OnAfterPopulate that is not provided in the RO Dataset. Instead, use the OnAfterOpen event.

• Update Rules. Update Rules are a great feature of RO/DA but, when I used them, I found it to be an all-or-nothing approach. My advice is to have two different schemas. One that does use Update Rules and where all Datasets use them and the other that doesn't.

• QuickReports (2). Not all conversions to QuickReports are successful. I discovered that the group break expressions failed. I found no successful resolution to this and re-provided the reports using ReportBuilder.

APPENDIX

The appendix contains three sections:

• Moving AstaClientDataSets to RO Datasets
• Details of ACDS Properties and Events that need to be re-provided
• Action to be taken in the event that there are Persistent Fields to convert

Moving AstaClientDataSets to RO Datasets

At the client:

1. Extract the SQL statement from the AstaClientDataSet (ACDS). Examine the dataset and complete the form provided in the next section (Details of ACDS Properties and Events that need to be re-provided).

At the server:

1. Select server project.

2. Create a Dataset in the Schema based on the SQL property in the ACDS.

3. Recompile server and launch as executable.

Back to the Client:

1. Select client project.

2. Select your remote ClientDataModule (CDM) and run the CDM wizard. Select DataTable wanted.

3. When created, cut the DataTable and DS to the correct location (target unit).

4. Rename the ClientDataSet (CDSXxx) to have the same name as the ACDS but leave off the leading 'A'.

5. Add the CDM (which contains the Adapter and RemoteService components) to your uses clause.

6. Re-apply Adapter and RemoteService properties to DataTable. Check the LogChanges property to ensure that it is True for editable datasets and false for read-only datasets.

7. Move all DS based events.

8. Check for persistent fields. If found, see the later section: 'Action to be taken in the event that there are Persistent Fields to convert'.

9. Move all table based events.

10. Find data controls that used the original DS and use the new DS.

11. Delete old DS and rename new DS to old DS name.

12. Delete old ACDS.

13. Check for ClonedCursor operations. If found:

• 13a. If original uses AfterPost and AfterDelete events to ApplyUpdates, assign those event handlers in code to the ClonedCursor as well.

14. Change all references from ACDS to CDS and compile.

15. Delete all Asta references.

16. Test the application.

Details of ACDS Properties and Events that need to be re-provided

This section is intended to be printed out and used as many times as required as you perform the process.

Fill in all of the details for each element of each AstaClientDataset (ACDS) that you intend to convert to an RO Dataset. Each column should contain a numerical value indicating the number of those items that need to be converted. The exception is the Editable column which indicates simply whether or not the data returned should be editable.

ACDS Name	Table Events	DS EVents	Persist Fields	Calc Fields	Lookup Fields	Field Events	SQL Params	Rewrite Where	Editable

As you work through the above, you'll be creating new entries in your Data Abstract schema. When you do this, record the name you used below to ease the transition:

ACDS Name	Mapping to Schema Name

Action to be taken in the event that there are Persistent Fields to convert

1. Check for bold fields (indicating changes from defaults). Copy properties where required.

2. If there are calculated fields:

• Set the fields' Calculated property to true.
• Set the fields' LogChanges property to false.
• If the DataType is string, set the Size property, and optionally the DisplaySize property. (Note that you must set the DataType property correctly to avoid design time exceptions when opening the DataTable).
• Move all field based events.

3. If there are lookup fields:

• Map properties like this:

ACDS	RO
DisplayWidth	DisplayWidth
FieldKind	No equal
FieldName	Name
KeyFields	KeyFields
LookupCache	LookupCache
LookupDataset	LookupDataset
LookupKeyFields	LookupKeyFields
LookupResultField	LookupResultField
Size	Size
	Lookup = true
	DataType = <set as reqd>