Miracle Open World 2011 - XML Index Strategies
1 like1,206 views
This document discusses strategies for choosing the right XML indexing method based on the type and structure of XML data. It describes options for indexing structured, semi-structured, and unstructured XML data stored as XMLType, including using the XMLIndex, Oracle Text index, and standard B-tree indexes. It provides syntax examples for creating different types of XMLIndexes and maintaining them over time.
Miracle Open World 2011 - XML Index Strategies
- 1. XML Indexing StrategiesChoosing the Right Index for the Right JobMarco Gralike
- 2. Strange Chainsaw Tree GuyOakTable MemberACE DirectorCustomer Advisory Board Member17+ years DBA, etc.Blog.gralike.comTechnology.amis.nl
- 3. Richard Foote (Mr. Index)OakTable MemberACE DirectorOracle Certified Professional22+ years DBA, etc.RichardFoote.wordpress.com
- 4. Refinement
- 5. Structured or Semi-Structured or…StructuredSemiStructured
- 7. Document Driven / Data Driven
- 8. Design: Width and Height and …13452XY6ZContent Height: minOccurs="0" maxOccurs=“unbounded”Content Width: type="xs:string“, restriction…?Content Distribution: histogram, statistics, skew, cardinality ?
- 9. XMLType “Under the Hood”DOM Tree ModelXQuerySQL/XMLXMLType AbstractionProcedural XQueryDB XQueryXVM (use “no query rewrite”)XQuery RewritePushdownSQL ExecutionXMLIndexStreaming XPath EvaluationRelationalAccess MethodsBinary XMLObject-RelationalRelational StorageSecure FilesSource OOW 2010 presentation S317428: Building Really Scalable XML Applications with Oracle XML DB and Oracle Text
- 16. XML Storage/Index Use CasesBinary XML(Schema based)XMLIndex Structured Component
- 18. XMLType Storage ModelsCLOBDefault until 11.2.0.2.0Non-Schema BasedBinary XMLOracle 11 and OnwardsSchema and Non-Schema BasedObject Relational (+Hybrid)Nested Tables, Types, Varray’sSchema Based
- 19. Storage Index Defaults (xmltype)Binary XML / CLOBLOB IndexObject RelationalDBMS_XMLSCHEMA “OPTIONS”Oracle 10g: Index Organized TablesOracle 11g: B-Tree Indexesxdb:annotationsStorage type xdb:SQLTypeStorage Typexdb:ColumnProps / xdb:TableProps
- 20. Index Methods (xmltype)CLOBOracle Text
- 21. XMLIndex (>= 11g)Binary XMLXMLIndex / Oracle Text
- 22. Virtual Columns (BTree)Object Relational Standard Indexes
- 23. Oracle TextIndex Methods (10.x)BTree IndexBTree IBTbookstoreOracle Text IndexFunction based Index (XPath)bookwhitepapertitleauthorauthorchaptertitleauthoridparagraphcontentcontent
- 24. Function-Based IndexDeprecated in 11.2Object Relational XMLType Storage(can be used, but shouldn’t, on CLOB)Performance wise the lesser option…SQL> CREATE INDEX function_based_index ON xml_data_table (extractValue(OBJECT_VALUE, '/Root/TextID'));
- 25. BTree / Bitmap Index (O.R.)Structured XML DataOrdered Collection Tables (OCT)ComplexTypes…“dot” notation using the “xmldata” pseudocolumnSQL> CREATE INDEX dot_notation_index ON xml_data_table ("XMLDATA". "TEXTID");
- 26. Syntax Index AlternativesSQL> CREATE INDEX function_based_again_idx ON xml_data_table (CAST("XMLDATA". "TEXTID“ as VARCHAR2(10)));SQL> CREATE INDEX oracle_11_applicable_only_index ON xml_data_tablexdt (XMLCast(XMLQuery('$i/Root/TextID' PASSING xdt.OBJECT_VALUE as "i" RETURNING content)as VARCHAR2(10)));
- 27. Oracle Text IndexUnstructured Data in XMLCLOB storage XML part in Object Relational XMLSecondary index on XMLIndexCan only index XML data TEXT nodesResult Set Interface (new in 11.2.0.2)Specify Query request and hit list requirements in XMLSQL> CREATE INDEX oracle_text_index ON xml_data_table (OBJECT_VALUE) INDEXTYPE IS CTXSYS.CONTEXT;
- 28. Index Methods (11.1)BTree IndexBTree IBTbookstoreSecondary Oracle Text IndexFunction based Index (XPath)bookwhitepaperUnstructuredXMLIndextitleauthorauthorchaptertitleauthoridparagraphcontentcontent
- 29. XML Document contains:Semi Structured Data and Structured DataSupports searching and fragment extractionWhen XPath queried is not known beforehandXMLType CLOB or Binary XML contentIf you use an XMLIndex and/or combine it with Structured XMLIndex(es)Usage: Unstructured XMLIndex
- 30. Simple: Unstructured XMLIndexSQL> CREATEINDEXxmlindex_idx ON “XMLTYPE_COLUMN"(xdata) INDEXTYPEISXDB.XMLINDEX;Index created.SQL> CREATEINDEXxmlindex_idx ON “XMLTYPE_TABLE"(object_value) INDEXTYPEISXDB.XMLINDEX;Index created.
- 31. Creating Unstructured XMLIndexCREATEINDEXXMLIDX ON XMLBINARY_TAB (object_value) INDEXTYPEISXDB.XMLIndexPARAMETERS ('PATHS (INCLUDE (/ROOT/ID /ROOT/INFO/INFO_ID ) NAMESPACEMAPPING(xmlns="http://localhost/xmlschema_bin.xsd") ) PATH TABLE path_table(TABLESPACE XML_DATA)PATH ID INDEX pathid_idx (TABLESPACE XML_INDX)ORDER KEY INDEX orderkey_idx (TABLESPACE XML_INDX)VALUE INDEX value_idx (TABLESPACE XML_INDX)ASYNC (SYNC ALWAYS) STALE (FALSE) ') PARALLEL LOGGING;
- 32. Path TablePATH TABLEPATH INDEX(PATHID, RID), BTREEORDER INDEX(RID, ORDER_KEY), BTREEVALUE INDEX(SUBSTRB("VALUE",1,1599))FUNCTION BASEDSECONDAIRY INDEXESUnstructuredXMLIndexf (x)Path Table
- 33. Unstructured XMLIndex (UXI)OnePath Table
- 35. Full Blown XMLIndex canbe BIG
- 43. Index Methods (11.2)BTree IndexBTree IBTbookstoreSecondary Oracle Text IndexFunction based Index (XPath)bookwhitepaperStructuredXMLIndexUnstructuredXMLIndextitleauthorauthorchaptertitleauthoridparagraphcontentcontentHighly StructuredIslands of Data
- 44. With highly Structured DataLikely candidates: ComplexTypesStructured Islands of DataCan be nested, but officially only one levelXMLTABLE “virtual” nested column hintWill create (multiple) “Content Tables” Multiple XPath defined same columns with different purposeThey deliver relational performance…!Usage: Structured XMLIndex
- 45. Simple: Structured XMLIndex“XMLTABLE” Driven SyntaxSQL> CREATE INDEX xmlindex_sxi on xml_data_table (xmlcol) indextype is xdb.xmlindex parameters ('GROUP employee_info_groupXMLTABLEEMP_CONTENT_TABLE ' '/employees/emp' ' COLUMNS empidNUMBER(10) PATH ' 'id' ' '); Be aware' '
- 49. NON Unique BTREE IndexYour ColumnsRIDrowidYOURcolumnXKeyRAWNot nullRIDrowidKeyRAWNot nullYOURcolumnsXCONTENT TABLE(s)YOURcolumnsXRIDrowidKeyRAWNot null
- 50. Structured XMLIndex (SXI)Content Table(s)BasedonXMLTABLEsyntaxXMLTable construct canbenestedbut:“Only ONE XMLType column allowed” VIRTUAL columnCanbemaintainedManuallySecondairyindexespossibleLOCAL parameter (partitioning)StructuredXMLIndexf (x)ContentTables
- 51. Adding Structured IndexesSQL> ALTER INDEX xmlindex_sxiparameters ('ADD_GROUP GROUP my_new_group XMLTABLE xml_content_tab_new ' '/root/extra' ' COLUMNS extracol VARCHAR2(35) PATH ' 'new_element' ' ');
- 52. Mixed XMLIndex OptionsbookstoreSecondary (text)IndexUnstructuredXMLIndexbookwhitepaperStructuredXMLIndexStructuredXMLIndextitleauthorauthorparagraphtitleauthoridchaptercontentcontent
- 53. Mixed XMLIndex structuresCREATE INDEX xmlindex on TEST_RANGE_XML (doc) indextype is xdb.xmlindexPARAMETERS (' PATH TABLE path_table PATHS (EXCLUDE(/root/ElementInfo)) '); BEGIN DBMS_XMLINDEX.registerParameter ('StructuredXML', 'ADD_GROUP GROUP ElementInfoXMLTABLExml_cnttable_valueinfo ' '/root/ElementInfo' ' COLUMNS ValueInfo VARCHAR2(100) PATH ' 'ValueInfo' '); END; /ALTER INDEX xmlindex PARAMETERS('PARAM StructuredXML');
- 54. XMLIndex MaintenanceALTER INDEXXMLIndex Parameter ChangesDBMS_XMLINDEX.DROPPARAMETERDBMS_XMLINDEX.MODIFYPARAMETERDBMS_XMLINDEX.REGISTERPARAMETERManual Synchronizing an XMLIndexDBMS_XMLINDEX.SYNCINDEX Pending Tables
- 55. “There Can Be Only One…”
- 56. Syntax AwarenessSYNC=ALWAYSMandatory when Combined XMLIndexSYNC=MANUALLockingSTALE=FALSE | TRUEHmmm…Empty XMLIndex tablesOOPS I got my “XMLTABLE” Syntax etc. “wrong”
- 57. Notes on XMLIndex (1)Only ONE XMLIndex is allowed in a user schemaAdd extra XMLIndex structures (structured or unstructured) via ADD_GROUP syntaxOnly SYNC=ALWAYS is allowed while using mixed XMLIndex structures or add more than one
- 58. Notes on XMLIndex (2)You need the LOCAL parameter to create local partitioned XMLIndexesAn XMLIndex on a HASH partitioned XMLType column or XMLType table, isnot (yet) allowedBut you can create an Oracle Text Index on such structures
- 59. RecapTrue understanding of Storage and Index options will provide:Optimal performanceOut perform XML (Java based)A lot of choice:Problems are ComplexAlso provides SolutionsGood design beforehand is the path to success
- 60. References (1)Oracle WhitepapersOracle XML DB : Choosing the Best XMLType Storage Option for Your Use Case (PDF)Oracle XML DB : Best Practices to Get Optimal Performance out of XML Queries (PDF)Bloghttp://blog.gralike.com (Dedicated XMLDB blog)Semi-Structured XMLIndex sectionStructured XMLIndex section
- 61. References (2)Oracle Open World Presentationon XML DBS317428: Building Really Scalable XML Applications with Oracle XML DB and Oracle TextXML DB OTN / FAQ Threadhttp://forums.oracle.com/forums/forum.jspa?forumID=34 http://forums.oracle.com/forums/thread.jspa?threadID=410714
Editor's Notes
- #5: It all comes down to packaging
- #6: Definitions of Structured, Semi-Structured and Unstructured data
- #9: Emp/Dept tables, Foreign/Primary Keys…Showing here ONLY 1 XML document…
- #10: See also OOW 2010, S317428: Building Really Scalable XML Applications with Oracle XML DB and Oracle Text – Nipun Agarwal, Oracle