Managing electronic records business objects using XForms and Genericode at the National Archives and Records Administration Balisage 2009 August 11-14, 2009 Quyen L. Nguyen National Archives and Records Administration quyen.nguyen@nara.govBetty HarveyElectronic Commerce Connection, Inc. harvey@eccnet.com Motivation Business Requirements System Architecture XForms and Genericode Approach Benefits Design and Implementation
Motivation Business and Technical Perspectives: ERA: fulfill NARA mission in the information technology age Provide long term preservation and access to digital objects Archivists and Agencies Record Managers will have a modernized system with automatic workflow that can streamline the digital archive business process. SOA Framework and XML in ERA System System Challenges
Evolution Relativity
Variety of domains and data types. Rapid volume growth. ERA Non-Functional Requirements Evolvability New record types, data types, and services could be added New technologies in software and hardware could be inserted Scalability. Ability to adapt to the growth of record volume and user community. User-friendly Secure Highly available Open standards Archival Business Object (ABO) Requirements Elimination of free text form. Structured fields that conform to a pre-defined data model. System validation of input data. Elaborate validation based on embedded business rules during creation/update of business objects. Presentation of ABO. Browser View: W3C HTML standard browser. Print View: XML --> XSL-FO --> PDF CRUDVS (Create, Retrieve, Update, Delete, Versioning, Search) operations will be supported. Storage model: XML Format Authority Lists Controlled vocabulary Controlled enumerated values Authority List Management Maintainability Responsible organization Building queries from various forms of search terms There can more than 20 authority lists in one ABO form Simple Authority List Name-value pairs:
Name Value/Definition
NARA National Archives and Records Administration
USPTO United States Patent and Trademark Office
NOAA National Oceanic and Atmospheric Administration
... ...
Complex Authority List More than a single name-value pair This table can be considered a complex code list because it contains values for the same information Paper Name.
Container Code Description Depth Width Height TotalCubicFoot
ACF-S Archival Quality Cubic Foot Container, Standard 16.500 12.500 10.750 2217.188
AER-A Aerial Film Can, Standard 5.500 5.500 10.000 302.5
AER-B Aerial Film Can, 500 7.250 7.250 9.500 499.3438
Business Rules Required or Optional data. GAO Concurrent Status Inter-dependency between fields Agency Manual Indicator=yes, then: Manual Name, Manual Version, Item ID within Manual. Authority List filtered based on another field Calculation performed based on some fields being selected Editable field based on state Workflow
TR Workflow
BPM workflow System orchestration SOA Framework
XML at: Business level Communication level Storage model In Search of an Approach for e-Forms How to ensure data integrity? Validate input data How to facilitate data exchange between components and services? SOA framework Data conversion Performance: response time and user experience Consistency Uniform and consistent error handling and error messages Modularity and reuse Low cost maintenance Modality of User-interface Storage Model Standard support In Search of an Approach for e-Forms and Code Lists Separate evolvability Separate modularity Separation of control Alternative Approaches for e-Forms <html> <body> <form action="createTR.php" method="post"> Transfer Request ID: <input type="text" name="tr-id" /> Agency: <input type="text" name="agency" /> <input type="submit" /> </form> </body> </html> <form action="createTR.jsp" method="post"> <form action="createTR.asp" method="post"> Alternative Approaches: Discussion
Benefits
Can design with MVC Leverage J2EE, .NET Leverage power of procedural language for business logic Aternative Approaches: Issues How to add or change a field? Changes required throughout all layers of MVC Development cost Turn-around time How to format field or section to make it attractive? Screen refreshes upon value selection in authority list How to represent values in a code list? Relation with XML data Marshalling of XML data Modification to schema requires code recompilation Aternative Approach for Authority Lists Use xsd:enumeration Code list changes, schema changes Schema version How to implement complex code lists? Use relational database SQL in applications Maintenance issue Data exchange requires conversion Problem Summary Evolvable design Extensible Adaptive to changing business rules Cost effective Fit into ERA SOA architecture XML standard based
Brief Introduction to XForms This section will provide a brief introduction into the XForms specification and the benefits of using XForms for creating business documents. XForms Specification XForms is a W3C Specification Version 3 - dated 29 October 2007 Version 2 - dated14 March 2006 Version 1 - dated 14 October 2003 Benefits of XForms Described as ext generation of webforms Standard Vocabulary Reusable modules (XInclude) Native XML Results Use of MVC (Model-view-controller) - separation of business logic and user interface. Used with other vocabularies, usually XHTML but could be custom vocabulary. Behaves as an HTML form. Complexity is hidden from user. XForms Controls Very similar to HTML forms but with more functionality.
XForms
Control		 Definition Example
<xforms:input> Free form text. Link to Example
<xforms:secret> Used to capture data while being hidden. Can be used to capture password information. Link to Example
<xforms:textarea> Free formed block of text in a multiline format. Link to Example
<xforms:output> Display information to the user. Link to Example
<xforms:upload> Provides the ability for a user to upload an external file. Link to Example
<xforms:range> Provides the ability for a person to upload a value from a range of values. Link to Example
<xforms:trigger> Provides a user triggered action Link to Example
<xforms:select> Provides the ability to allow the user to select multiple values from a list. Link to Example
<xforms:select1> Provides the ability to allow the user to select a single value from a list. Link to Example
<xforms:submit> Provides the ability submit data to a processor. Works in conjunction with the <xforms:submission> element. Link to Example
Genericode Introduction What is Genericode OASIS-Open Specifcation
http://docs.oasis-open.org/codelist/cs-genericode-1.0/doc/oasis-code-list-representation-genericode.pdf Specification for encoding code lists in XML XML W3C Schema
Genericode and XForms Genericode Structure Three major sections: Identification Section Column Setup Section The body of the code list.
CodeList Structure
 Sample Genericode Holdings_Measurements 1 US National Archives Records Administration 1 Container Code Container Description Depth Width Height Total Cubic Footage ACF-S Archival Quality Cubic Foot Container, Standard 16.500 12.500 10.750 2217.188 AER-A Aerial Film Can, Standard 5.500 5.500 10.000 302.5 AER-B Aerial Film Can, 500 7.250 7.250 9.500 499.343 ... ]]> Genericode and XML Fat What is XML fat? Layers unnecessary of XML hierarchy for the application layer? Unnecessary is in the eye of the beholder. Genericode provides a 'standard' format to create, maintain and update code lists. There are arguments for and against direct use of Genericode in the application layer. Agree not to argue amonst ourselves about the BMI (Body Mass Index) of Genericode Let's Move Forward Reduced Fat Code List Approach In many cases the fat content of the genericode is not a concern. However when organizations have multiple codelists and hundreds of codes for a single code list cutting the fat is essential for streamlined forms. Developed a XSLT Convert Genericode to 'fat free' version Fat-free version less than 25% size of Genericode. (Paper example 87 lines vs. 22 lines) Expedites processing of code list. Code names located in elements vs. attributes. Conversion to Form Code List
Conversion To Form Code List
 Fat Free Version of Paper Code List Example ACF-S Archival Quality Cubic Foot Container, Standard 16.500 12.500 10.750 10.750 AER-A Aerial Film Can, Standard 5.500 5.500 10.000 10.000 AER-B Aerial Film Can, 500 7.250 7.250 9.500 9.500 ... ]]> Incorporating Form Codelist into XForm XForms <xforms:instance> element can point to a file. Can point to a fully-qualified URL Point to the local file system Fully Qualified URL for Code List The two examples below are both fully-qualified URL's. The first example points to the eXist XML repository that contains the codelists. The second example points to a fully qualified URL. In the example below the <xforms:instance> element points to the filesystem. The directory root is the directory where the XForms is located.
Full URL
]]> Code List Located on the File System A code list can reside on the file system. The XForms engine will require access to the file system.
Pointing to the File System
<xforms:instance id="AccessRestriction-instance" src="CodeLists/ContainerFormCodeList.xml"/> Using the Code List Select1 Create an <xforms:select1> control. The <xforms:select1> control creates a radio button selection. The 'nodeset' attribute is the fully-qualified XPath statement to retrieve a list of values from the codelist. Notice in the sample below, the <forms:label> and <xforms:value> are the same. The label displays the value of the PaperCode.
Example of Select1 for Container Type
<xforms:select1 ref="era:ContainerType"> <xforms:item> <xforms:label>Choose One...</xforms:label> <xforms:value/> </xforms:item> <xforms:itemset nodeset="instance('Container-instance')/Row/ContainerDescription"> <xforms:label ref="."/> <xforms:value ref="."/> </xforms:itemset> </xforms:select1>
Select1 Display
Only 1 value can be saves. Repeating Codes If you have repeating values, you should use <xforms:repeat> to wrap each code into XML. <xforms:repeat nodeset="era:ContainerInformation/era:Container" id="repeatContainer"> <xforms:select1 ref="era:ContainerType"> <xforms:item> <xforms:label>Choose One...</xforms:label> <xforms:value/> </xforms:item> <xforms:itemset nodeset="instance('Container-instance')/Row/ContainerDescription"> <xforms:label ref="."/> <xforms:value ref="."/> </xforms:itemset> </xforms:select1> </xforms:repeat> <xforms:trigger> <xforms:label>Repeat Paper Code</xforms:label> <xforms:insert nodeset="Paper" at="index('papercode-repeat')" position="after" ev:event="DOMActivate" /> </xforms:trigger>
Display of Repeating Codes
 Populating Values From Single Code The <xforms:bind> element is uses to bind values from the codelist instance to the resulting XML document. From the code <PaperCode> we want to populate three values. We will need 1 <xforms:variable> and three <xforms:bind> elements.
Paper Code List
<?xml version="1.0" encoding="UTF-8"?> <FormCodeList codeListName="Holdings_Measurements" version="1"> <Row> <ContainerCode>ACF-S</ContainerCode> <ContainerDescription>Archival Quality Cubic Foot Container, Standard</ContainerDescription> <Depth>16.500</Depth> <Width>12.500</Width> <Height>10.750</Height> <TotalCubicFoot>10.750</TotalCubicFoot> </Row> . . . </FormCodeList> Setting the <xforms:variable> element The <xforms:variable> element is used to provide the capability to compare the instance value (form) against the codelist value. The codelist can reside in the database or on the filesystem.
<xforms:variable> attributes
nameThe name you give the variable. This will be the name you use to reference the name in a calculation. selectThe XPath location to location value you want to set the variable too.
Set XForms Variable
<xforms:variable name="containerType" select="era:ContainerInformation/era:Container/era:ContainerType"/> <xforms:setvalue ref="era:ContainerDimensions/era:Height" ev:event="DOMActivate"/> Setting <xforms:bind> The <xforms:bind> element must be used for every value in the codelist that gets set.
<xforms:bind> attributes
nodesetThe XPath statement for the element and/or attribute being set. calculateThe XPath for determining the value that should be populated. Notice a variable is used to compare against. [ContainerDescription = $containerType]
XForms Binds
<xforms:bind nodeset="descendant::era:Height" id="bindHeight" calculate="instance('Container-instance')/Row[ContainerDescription = $containerType]/Height"/> <xforms:bind nodeset="descendant::era:Width" id="bindWidth" calculate="instance('Container-instance')/Row[ContainerDescription = $containerType]/Width"/> . . . XForms-Genericode Demo
Demo of XForms-Genericode In the words of Andy Kaufman
Thank You Very Much! The End