DAML (OntoAgents) Homework Assignment 3:
Large-Scale DAML Content

1. We have generated a DAML/RDF Version of Wordnet. The RDF source of Wordnet and a schema describing the ontology used to describe the content of Wordnet is available at: http://www.semanticweb.org/library

2. We have also generated 10.820 statements (according to DAML viewer) so far by crawling the URLs: https://www.daml.org/homework/1/summary.htmll and http://www.ontoknowledge.org/oil/case-studies/ as entry points. The statements including are "ontological" and "instance oriented" as well, consisting of pages like the CIA factbook , the KA2 Ontology and the DAML pages of Homework Assignment 1 .

The result page of this crawl can be found at http://www.aifb.uni-karlsruhe.de/WBS/daml/homework/crawlstatements.rdf (670 KB)

3. Lessons Learned. Techniques and Tools

For 1: Datasets of this size need database support - otherwise no use is possible.

For 2: We used the RDF/DAML-Crawler which writes a repository of RDF facts obtained during crawling and verify this repository with the DARPA DAML-Viewer . We use the DAML-Viewer also for counting the amount of statements in the repository file (file->properties).

During the crawling some questions arised:

• how to avoid to get the same page twice? The crawler have to compute an hash value for the file content in order to detect the same web page
• Ther are problems with incorrect RDF statement when crawling, some are not valid or includes missing lines
• Some have an improper use of Namespaces
• or bad data in the model, contradictory facts in the model => need for an DAML validator
• The statements are not only expressed in english i.e. there are facts in french or german as well. The OntoAgents will need a translation service for that.
• How to ensure the consistent of the statements in the crawled repository,
• how to avoid multiple occourence of the same statement, when crawling?
• there should be stored the origin of the statement, the model where it comes from
• =>need for an DAML statement storage system