This project is based on a paper called Building Logical Internetwork Data Warehouse Using Distributed Object Technologies (DOT) published in August 1997. A similar implementation is already done using C++, C and REXX. Reading the paper is useful in order to understanding how an existing system works, but students should keep an open mind in designing and implementing the project.
Our development system can be used to structure distributed software systems with decoupled components connected to data processors or databases. The system divides a distributed system into four basic components. Network Object is responsible for coordinating communication and data access. Interface Object displays information to the user. The goal of the optional component Logic Object is to provide a consistent workflow and logic flow for Interface Objects. Virtual Object Registry provides services such as authentication, dynamic version control and other control information for both Network Objects and Interface Objects.
Depending on how big the group is, students should implement all or some of the components of the development system. Instead of using command base Network Object and Interface Object generators mentioned in the paper, GUIs implemented using swing should be provided. The development system should provide rapid development of Web applications or Java cross platform applications. The Network Object GUI generator should gather information (such as dbnames, userids, passwords, driver names, meta data query or specific table information, etc) from the users and generate Java applet or application with selected functions (such as select, add, execute) to targeted database. The Interface Object GUI generator should generate high level Java GUI to work with Network Object. For example, all the required fields to a selected table and radio button for running different Network Object functions should be generated.
Test Case
Since this project is to implement a development system, our test case should be using our project system to implement a product. Thus, the product we implemented using our development system is our test case !
Suppose we are developing an Asset Management Information System (AMIS) designed to run on a wide area networks. There are three different types of users. Users will have the ability to display his/her asset information, for example, what type of machines the user owns, if the machines are on the networks, who will be the network administrators, what type of network system is available (ATM, Token Ring, Ethernet), which cable or outlet is available, etc. Users should also be able to "certify" the machines they own, this will prove the ownership of a certain machine. Additional functions such as "Add Machine" and "Return Machine" should also be provided. Managers will be interested in what types of machines are owned by the department, how much will be the total cost, who is responsible for each machines, etc. Asset administrators will be interested in asset management functions such as tracking of sensitive parts, databases administrations, machines relocation processing, etc.
All the front-end of the system should support on-line retrieval of data information from different data processing machines and display the information using graphical user interface. Users are using different platforms (Windows 95, Windows NT, OS/2, VM, UNIX, etc.) of front-end when accessing the information. As the system will be used by a large variety of employee, all the front-end (user interface) should be consistent and easy to use. This will lower production cost by reducing end-user training and software support.
The backend of the system will be connected to different data processing machines. The data processing machines can be databases on different platforms from different vendors, or legacy application exports data from flat files.
Requirements
This development system should be able to balance the following forces (requirements):
Design Overview
Classic technique "Divide and Conquer" can be used to divide our development system into server-side processing and client-side processing. Network Object is used for server-side processing, whereas Interface Object and Logic Object are used for client-side processing.
Network Object provides transparent service for communication as well as data/database access. As with any object, a Network Object contains both data and operations on the data. The object itself is actually logically exists in the network. That means that a Network Object can "reside on" and "be accessed by" any system connected to the network. Thus, the Network Object hides communication and data/database implementation specific details from the users and services. In other words, Network Object can be view as a Java/JDBC program that can be called by any client.
Interface Object and Logic Object provide a set of consistent user interfaces across different platforms. Virtual Object Registry is a general service provider for Logical Data Warehouse components. Logical Data Warehouse components can be dynamically changed, added, deleted and relocated with the use of Virtual Object Registry. Services for Network Objects include authentication, dynamic version control, access to control information for dynamic configuration (protocol, memory, CPU), etc. With the use of Virtual Object Registry, Network Objects can be mirrored to each other without exposing the mirroring information to users.
Services for Interface Objects include client-side authentication, dynamic virtual desktop environment setup, tracking of access information, etc. During the startup of a Interface Object, it can contact the Virtual Object Registry for customization information. The Virtual Object Registry will return different information based on the access level of the users. Thus, with the combination of Virtual Object Registry and Interface Objects, run-time user interface customization can be achieved.
Structure
Our development system defines the following participating components and sub components:
Configuration Board
Network Objects
Server-side Proxy
Data Info Objects
Interface Objects
Logic Objects
Bridge
The Virtual Object Registry is the main component of the development system. It implements central services such as communication, resource handling, access control and authentication between Network Objects and Interface Objects. Other components will register directly or indirectly with the Virtual Object Registry. They do this by using a standard interface between components.
The Configuration Board is a sub component of the Virtual Object Registry, it allows different components (Network Object or Interface Object) to read and write information via standard interface. The information written in the Configuration Board will be used as run-time consulting information. For examples, an Interface Object and a Network Object can write out their maximum memory allocation in the network application layer when the connection is first established. Then, by consulting and updating the memory information in the Configuration Board, the components can send and receive data in a most efficient way. Thus, with the use of Configuration Board, instead of picking some default number, say 64K bytes, the ideal memory allocation number can be used. If the ideal number is 640K bytes, the components can send data in one 640K bytes chunk instead of ten 64K bytes chunk, a saving of 9 network calls.
The Network Object is the component that is responsible for coordinating communication between itself and Interface Objects. The naming and location of the Network Objects are provided by Server-side Proxy. Network Object also provides hidden data access for Interface Objects. It achieves its goal by exporting standard interface for Interface Objects to access data defined in Data Info Objects. When a Network Object is created, it will register with the Virtual Object Registry as well as the Configuration Board. By this registration, the Network Object will be able to provide intelligent service such as protocol negotiations (SPX to SPX2 to TCP) and network application memory allocation to the Interface Objects. For this project, TCP can be assume as the main network protocol.
The Server-side Proxy is a service provider for naming and location services for Network Objects. It will store the naming and location information for Interface Objects to access. Thus, this component provides location-transparent service invocations. Virtual Object Registry also uses this component to implement Network Objects mirroring for the system.
The Data Info Object is a sub component for Network Objects. It contains methods on accessing data in which Network Objects are interested. On the one hand, it can contains methods making use of SQL to access different kind of databases. On the other hand, it can contains methods that will invoke APIs of legacy application for accessing data. There is a one-to-one or one-to-many mappings between Network Object and Data Info Object. From a data access point of view (Interface Object point of view), the Data Info Object is well hidden within the Network Objects. Client just send a message to the Network Object and the Network Object will return data associated with it.
The Interface Object is the component that will present consistent user interface to access data in multiple platforms. It will access data to be presented by communicating with the Network Objects through standard interface.
The Logic Object is the helper component for Interface Object and Virtual Object Registry. When the logic or work flow of the application can be commonly grouped together, the Logic Object can be used. This is an extra credit component for the project.
The Bridge is a linking component between Network Objects and Interface Objects. It enables the sharing of common information between Network Objects and Interface Objects. It can take the form of data dictionary (for data fields needed to be accessed) and shared between Interface and Network Objects. Or it can take the form of shared memory or remote procedure calls for linking between Network Objects and Interface Objects. In our implementation, we will employ code generators, and the linking between Network Object and Interface Object will be automatically generated. In other words, the Bridge will become a build-in component of the generated code.
Scenarios
This section focuses on the most relevant scenarios in the operation of our development system.
Scenario I illustrates the behavior when an Interface Object (front-end) request information to be displayed to the users:
The Bridge request location information from NetObj Proxy.
After received location information from NetObj Proxy, Bridge forwards the processInfo message to the correct Network Object.
The Network Object extract data internally from Data Info, and send a processResponse message to the Bridge.
The Bridge forwards the processResponse message to the Interface Object.
Scenario II illustrates the behavior of the Interface Object during initialization. This scenario denominates the creation of the "virtual desktop" depending on the user’s access control (Please note that in this case, the user information is stored in the Configuration Board):
The Virtual Object Registry receives the information, checks the user ID’s access authority and requests configuration information from the Configuration Board.
The Configuration Board returns configuration information based on the user ID.
The Virtual Object Registry requests the initialization of the desktop to be run by Logic Object. For example, if this user is an administrator, the Logic Objects will run the user interface (Interface Object) initialization for administrators. As a result, a few more administration functions will be available for the user interface.
Customized user interface is run as a result.
Scenario III illustrates the initialization of the desktop when the configuration data is stored in the network. This example illustrates the flexibility of virtual desktop implementation:
The Virtual Object Registry receives the information, checks the user ID’s access authority and requests configuration information from the Configuration Board.
The Configuration Board returns results indicating that the configuration information is stored in the network interfacing with a Network Object.
The Virtual Object Registry requests the location information of the Network Object from the NetObj Proxy.
After obtaining information on the Network Object, the Virtual Object Registry requests the configuration data from the Network Object.
The Network Object extracts data from the DataInfo Object. The data can actually be from a database or from legacy application. The detail is hidden from the Virtual Object Registry.
Configuration information is returned to the Virtual Object Registry.
The Virtual Object Registry requests the initialization of the desktop to be run by Logic Object.
Customized user interface is run as a result.
Implementation Discussion
This session discuss how to use our development system to implement our AMIS product:
Identify the server-side processing elements. Identify all the data processors / databases available in the application domain. The data processor can be legacy application which will export data after processing, or databases from different vendors. In the case of AMIS, employee information database, cable database and inventory accounting database are needed.
Decide which data fields needed to be accessed by Network Objects in each data processor / database. Not all the fields in a database will be needed by the AMIS system. For example, employee name, office location and serial number will be needed from the employee information database, but not the salary information.
Implement the Virtual Object Registry and the Configuration Board sub component. In the case of AMIS, the Virtual Object Registry is implemented with a Java program with access control information and other control information stored in Andrew File System (AFS token can be used for user authentication).
Use the Network Object Code Generator to generate code for different Network Objects.
The input of the Network Object Code Generator will be a four generation (or
script) language. The following type of files are generated:
- DDS Files (Data definition files for optional database creation)
- IDL File (Definition files for remote procedure call )
- Client Code (Stub code for network communication)
- Server Code (Stub code for server network communication)
- Procedure Code (Code contain SQL statement andoperations).
(The above are very general terms, in the case of using Java, the appropiate terms should be used.) As most of code is generated, it is relative easy to implement new technologies within the Network Objects. For example, if we upgrade from internal remote procedure call to a new version of DCE remote procedure call or JDBC, all we need to do is to regenerate the code for update. The same scenario applies if we want to use JDBC for database access instead of imbedded SQL.
Identify all the Front-ends (user interfaces) needed. In the case of AMIS, User,
Administrator and Managers are needed. The following figure shows the
administrator menu of the AMIS:
Generates Interface Objects. With control information inputted from the Network Objects, use the Interface Object Code Generator to generate user interface code for the system. The input to the Interface Object Code Generator will be a four generation language with a choice of imbedded language (e.g. C, C++, Java, REXX). The basic user interface will be generated in multiple platforms, and the imbedded language will be ported as well. Since we input control information (e.g. data dictionary) from the Network Objects during the generation of Interface Objects, the linking between Network Objects and Interface Objects is automatic. In other words, the Bridge component of the Logical Data Warehouse is implemented (generated) under the cover.
The following two figures show that even though the front end of the AMIS system will be run on totally different platforms, the look and feel can be kept consistent and "platform friendly".
Interface Object on UNIX platform:
The same Interface Object on the WWW:
Implement the Server-side Proxy. In the case of AMIS, the Server-side Proxy is implemented as a Network Object with data information on naming and location of other Network Objects (Good software reuse !). The Virtual Object Registry can be set to always talk to this particular Network Object for naming and location information.