CSE 636: Data Integration

Fall 2008 (#486324)

What's New

Please check the newsgroup for announcements. Lecture notes will be posted on this web page. A substantial part of the site will be in HTML and PDF formats.

Staff

• Instructor: Dr. Michalis Petropoulos

Newsgroup

sunyab.cse.636

Course Description

Data integration has been recognized as a research topic of big practical importance. The availability of integrated data from multiple independent, heterogenous data sources is crucial for many applications. Data integration requires combining and matching information in different sources, and resolving a variety of conflicts. XML is becoming a de facto data integration standard. With the number of data sources growing very quickly, data integration is bound to become even more important in the future. This course will survey selected theoretical and practical issues arising in data integration.

The course is based on the instructor's lectures and guest lectures. The lectures cover most of the material in the course.

Schedule

The following is a tentative schedule of lectures. Changes will be posted to the newsgroup.

Since the lectures are being recorded, here is the link for the videos.

Prerequisites

An introductory database course equivalent to CSE 560 or CSE 562

Text

None required. A list of recommended books goes as follows:

• Database Systems: The Complete Book
  by Hector Garcia-Molina, Jeff Ullman, and Jennifer Widom
• Database Management Systems
  by Raghu Ramakrishnan and Johannes Gehrke
• Fundamentals of Database Systems
  by Ramez Elmasri, Shamkant B. Navathe
• Foundations of Databases
  by Serge Abiteboul, Richard Hull, Victor Vianu
• Data on the Web
  By Abiteboul, Buneman and Suciu

Grade Computation

• Assignments: 15% (set of 3, 5% each)
• Final: 20%
• Project: 60%
• Participation: 5%
• Grades will be posted here...

Problems

• DTD/XML Schema Practice Problems
• Assignment 1
• Practice problem on GAV-style integration
  • Source A: Schema - Data
  • Source B: Schema - Data
  • Integrated Schema
  • Attempt 1: XQuery - Data Union, Restructuring
  • Attempt 2: XQuery - Data Duplicate Elimination, AntiSemiJoin
  • Attempt 3: XQuery - Data Nested AntiSemiJoin
• Assignment 2 (Solutions)
• Assignment 3 (Solutions)
• SchemaSQL Practice Problem

Project

• Phase 1 Specification (Updated on Friday, November 7th)
  1. Deliverable 1
  • Here is the JavaCC Parser Generator web page. Here is a very simple XQueryParser you can use as a starting point, and here is a test XQuery expression.
  2. Deliverable 2
  • Here is a skeleton of an XPathProcessor you can use as a starting point, and here is a test XPath expression and a test XML file.
  • To access XML files, you should use the standard DOM interface. There is a number of XML DOM parser implementations. We recommend the DOM parser that comes with Java (1.5 and above). The relevant packages are:
    • javax.xml.parsers and
    • org.w3c.dom
    The W3C spec of DOM is located here but we believe it is write-only (nobody may have been able to fully read it). The JavaDocs for tha above mentioned packages are a good reference, but the easiest way to get started with DOM is to walk through the DOMPrinter sample program that uses these packages and Sun's DOM Tutorial.
  • For test data, download Shakespeare's play, Julius Caesar, in XML form. Here is the associated DTD and here is a test XQuery expression.
• Phase 2 Specification (Updated on Friday, November 14th)

Resources

• XML 1.0 W3C Recommendation
  Includes DTDs.
• Document Object Model (DOM) W3C Recommendation
• XML Tutorials ( ZVON, W3Schools )
• DTD Tutorials ( ZVON, W3Schools )
• Online DTD Validator

• XML Schema W3C Recommendation
  Consists of three specifications. Start with XML Schema Part 0: Primer.
• XML Schemas: Best Practices
• XML Schema Tutorials ( ZVON, W3Schools )
• Validate an XML file with respect to a DTD or an XML Schema using standard Java. Here is an XML Schema and an XML file you can use for testing. In the example XML file, note how you specify the XML Schema that it conforms to.
• Another Online XML Schema Validator

• XML Query (XQuery) W3C Candidate Recommendation
  Consists of multiple specifications. Includes XPath. Start with XQuery 1.0: An XML Query Language.
• XPath Tutorials ( ZVON, W3Schools )
• XQuery Tutorials ( IPEDO, W3Schools )
• AltovaXML
  Implementation of the W3C XQuery standard. Provides both command line and languages-specific interfaces.
• Saxon
  Open Source XQuery processor developed by Michael Kay.
• XQilla XQuery Engine
  Open Source XQuery processor developed by Oracle.

• Web Services Description Language (WSDL) 1.1 W3C Note
• W3C Web Services Activity
  Consists of multiple groups. Includes WSDL and SOAP, among other specifications.
• Web Services Tutorials ( BRICS, W3Schools WSDL, W3Schools Web Services )
• A Busy Developers Guide to WSDL 1.1
• Apache Axis
  Java implementation of SOAP.
• Apache Axis Tutorial (PDF)
• Apache Tomcat
  This is the J2EE-compliant application server you will use for the project.
• The J2EE 1.4 Tutorial
  This tutorial covers many web technologies, including web application configuration and Java servlets.
• The Java Web Services Tutorial
  This tutorial covers many web services-related technologies.
• XMethods
  A collection of information about existing Web services.
• XMethods' WSDL Validator
  Make sure your WSDL makes sense. Your WSDL must be accessible over the web, i.e., you must provide a WSDL URL.

• Datalog Educational System (DES) is a basic implementation of Datalog with recursion and stratified negation.

Rules & Policies

Exam Rules

Group study and discussion are encouraged, but exams must be your own work. For coding assignments, if you use a piece of code which you borrowed from elsewhere and therefore did not write yourself, make sure you comment it to show this. Zero tolerance on plagiarism/cheating: consult the University Code of Conduct for details on consequences of academic misconduct, and see also the academic integrity policy of the CSE department.

Grading Policies

Write clear arguments. Be neat and precise. Getting the right answer may not be enough. The derivation and quality of writing counts! Don't write many different things in hope that you'll get the points if one of them is the right one. Indeed, you will lose points if you follow such a policy.