Difference between revisions of "Cube DSL"

Latest revision as of 19:22, 4 April 2017

Introduction

MDX and OLAP Cube

MDX(MultiDimensional eXpressions) is a query language for OLAP (OnLine Analytical Processing) databases, much like SQL is a query language for relational databases. It is also a calculation language, with syntax similar to spreadsheet formulas.

SQL was designed to query three-dimensional data structures, called tables, where data are organized in rows, columns and values. In OLAP, data are organized around multiple measures, dimensions, hierarchies, and levels.

Similar to the way tables and columns are central to SQL, dimensions, hierarchies, and levels are the centerpieces of MDX. In this way, business models are mapped to language-specific concepts (e.g. a list of countries will be mapped as an MDX dimension). Measures in MDX are a special case of dimensions and they hold numeric values. The concrete value of a dimension (e.g. Paris) is a member of the dimension and is similar to the value contained in an SQL table.

An OLAP Cube is a collection of dimensions indexing a list of measures. For example, a geometrical cube can have three dimensions, whereas a logistical "cube" can have many more dimensions.

More information can be found at:

http://en.wikipedia.org/wiki/MultiDimensional_eXpressions

http://en.wikipedia.org/wiki/OLAP_cube

Mondrian

Mondrian is an open source OLAP server, written in Java. It supports the MDX query language and XML for analysis and olap4j interface specifications. It reads from SQL and other data sources and aggregates data in a memory cache.

Mondrian is used for:

• high performance, interactive analysis of large or small volumes of information
• dimensional exploration of data, for example analysis of sales by product line, by region, by time period
• parsing the MDX language into SQL to retrieve answers to dimensional queries
• high-speed queries through the use of aggregate tables in the RDBMS (Relational DataBase Management System)
• advanced calculations using the calculation expressions of the MDX language

Schema

A schema defines a multi-dimensional database. It contains a logical model, consisting of cubes, hierarchies, and members, and a mapping of this model onto a physical model.

The logical model consists of the constructs used to write queries in the MDX language: cubes, dimensions, hierarchies, levels, and members.

The physical model is the source of the data which is presented through the logical model. It is typically a star schema, which is a set of tables in a relational database.

Schema file

The schema file is an XML file that defines the dimensions and measures for the cube and maps them to the database tables. The product and location dimensions contain several levels each. Each level defines its corresponding field in the database table. The different values of the field become the members of the level.

Note: The levels must be specified in the intended order of breakdown.

• The structure of the XML document is as follows:
  

  <Schema>
     <Cube>
        <Table>
           <AggName>
              aggElements
           <AggPattern>
              aggElements
           <Dimension>
              <Hierarchy>
                 relation
              <Closure/>
              <Level>
              <KeyExpression>
                 <SQL/>
              <NameExpression>
                 <SQL/>
              <CaptionExpression>
                 <SQL/>
              <OrdinalExpression>
                 <SQL/>
              <ParentExpression>
                 <SQL/>
              <Property>
                 <PropertyExpression>
                    <SQL/>
           <DimensionUsage>
           <Measure>
              <MeasureExpression>
                 <SQL/>
              <CalculatedMemberProperty/>
           <CalculatedMember>
              <Formula/>
              <CalculatedMemberProperty/>
           <NamedSet>
              <Formula/>
           <VirtualCube>
              <CubeUsages>
                 <CubeUsage>
              <VirtualCubeDimension>
              <VirtualCubeMeasure>
           <Role>
              <SchemaGrant>
                 <CubeGrant>
                    <DimensionGrant>
                    <HierarchyGrant>
                       <MemberGrant/>
        <Union>
           <RoleUsage/>
     <UserDefinedFunction/>
     <Parameter/>
  

  
  
• relation ::=
  

     <Table>
        <SQL/>
     <View>
        <SQL/>
     <InlineTable>
        <ColumnDefs>
           <ColumnDef>
        <Rows>
           <Row>
              <Value>
     <Join>
        relation
  

  
  
• aggElement ::=
  

     <AggExclude>
     <AggFactCount>
     <AggIgnoreColumn>
     <AggForeignKey>
     <AggMeasure>
     <AggLevel>
  

  
  

Logical elements

Physical elements

Aggregate tables

Access control

Extensions

Miscellaneous

Cubes

A Cube is a named collection of Measures and Dimensions. The one thing the Measures and Dimensions have in common is the fact table. The fact table holds the columns from which Measures are calculated, and contains references to the tables which hold the Dimensions.

► Syntax:

<Cube name="{cube name}">
    <Table name="{table name}"/>
    ...
</Cube>

The fact table is defined using the <Table> element. If the fact table is not in the default schema, you can provide an explicit schema using the schema attribute, for example:

► Syntax:

<Table schema="{schema name}" name="{table name}"/>

You can also use the <View> construct to build more complicated SQL statements.
The <Join> construct is not supported for fact tables.

Measures

Each Measure has a name, a column in the fact table, and an aggregator.

► Syntax:

<Measure name="{measure name}" column="{column name}" 
    aggregator="{aggregator name}" datatype="{type name}" 
    formatString="{format}"/>

► Example:

<Measure name="Store Sales" column="store_sales" aggregator="sum" 
    datatype="Numeric" formatString="#,###.00"/>

The aggregators are

• sum - Returns the sum value of the specified set expression.
• count - Returns the number of elements in the specified set.
• min - Returns the minimum value of the specified set expression.
• max - Returns the maximum value of the specified set expression.
• avg - Computes the average value of the specified set.
• distinct-count - Returns the number of unique, non-empty elements in the specified set. It has some limitations if your cube contains a parent-child hierarchy.

The optional datatype attribute specifies how cell values are represented in Mondrian's cache, and how they are returned via XML for analysis.

The datatype attribute can have values:

• String
• Integer, default value for "count" and "distinct-count" measures
• Numeric, default value, except for "count" and "distinct-count" measures
• Boolean
• Date
• Time
• Timestamp.

An optional formatString attribute specifies how the value is to be printed. The ',' and '.' symbols are locale-sensitive.

A Measure can have a caption attribute to be returned by the Member.getCaption() method instead of the name. Defining a specific caption does make sense if special letters (e.g., Σ or Π) are to be displayed.

► Example:

<Measure name="Sum X" column="sum_x" aggregator="sum" caption="&#931; X"/>

Dimensions, Hierarchies, Levels

A member is a point within a Dimension determined by a particular set of attribute values. The gender Hierarchy has the two members 'M' and 'F'. 'San Francisco', 'California' and 'USA' are all members of the store Hierarchy.

A Hierarchy is a set of members organized into a structure for convenient analysis. For example, the store Hierarchy consists of the store name, city, state, and nation. The Hierarchy allows you to form intermediate sub-totals: the sub-total for a state is the sum of the sub-totals of all of the cities in that state, each of which is the sum of the sub-totals of the stores in that city.

A Level is a collection of members which have the same distance from the root of the Hierarchy.

A Dimension is a collection of Hierarchies which discriminate on the same fact-table attribute (say, the day that a sale occurred).

► Syntax:

<Dimension name="{dimension name}" foreignKey="{key name}">
    <Hierarchy hasAll="true" primaryKey="{key name}">
        <Table name="{table name}"/>
        <Level name="{level name}" column="{column name}" 
            uniqueMembers="true"/>
    </Hierarchy>
</Dimension>

► Example:

<Dimension name="Gender" foreignKey="customer_id">
    <Hierarchy hasAll="true" primaryKey="customer_id">
         <Table name="customer"/>
         <Level name="Gender" column="gender" uniqueMembers="true"/>
    </Hierarchy>
</Dimension>

This Dimension consists of a single Hierarchy, which consists of a single Level called Gender.

The values for the Dimension come from the gender column in the customer table. The gender column contains two values, 'F' and 'M', so the Gender Dimension contains the members [Gender].[F] and [Gender].[M].

For any given sale, the Gender Dimension is the gender of the customer who made that purchase. This is expressed by joining from the fact table "sales_fact_1997.customer_id" to the Dimension table "customer.customer_id".

► Syntax:

<Table schema="{schema name}" name="{table name}"/>

You can also use the <View> construct to build more complicated SQL statements. The <Join> construct is not supported for fact tables.

"ALL" member

By default, every Hierarchy contains a top Level called (All), which contains a single member called (All {hierarchyName}). This member is parent of all other members of the Hierarchy, and thus represents a grand total. It is also the default member of the Hierarchy; that is, the member which is used for calculating cell values when the Hierarchy is not included on an axis or in the slicer. The allMemberName and allLevelName attributes override the default names of the all Level and all member.

If the <Hierarchy> element has hasAll="false", the 'all' Level is suppressed. The default member of that Dimension will now be the first member of the first Level; for example, in a Time Hierarchy, it will be the first year in the Hierarchy. Changing the default member can be confusing, so you should generally set hasAll="true".

Time dimensions

Time dimensions based on year/month/week/day are coded differently in the Mondrian schema due to the MDX time-related functions:

• ParallelPeriod([level[, index[, member]]]) returns a member from a previous/next period with the same position in the level as the specified member.
• PeriodsToDate([level[, member]]) returns the set of members starting from the first (earliest) sibling of the specified member and ending with the specified member itself.
• WTD([member]) returns PeriodsToDate( Week_Level, Member ).
• MTD([member]) returns PeriodsToDate( Month_Level, Member ).
• QTD([member]) returns PeriodsToDate( Quarter_Level, Member ).
• YTD([member]) returns PeriodsToDate( Year_Level, Member ).
• LastPeriod(index[, member]) returns the Level members that stand before the specified member, and including the specified member itself.

Time dimensions have type="TimeDimension". The role of a Level in a time dimension is indicated by the Level's levelType attribute, whose allowable values include:

► Example:

<Dimension name="Time" type="TimeDimension">
    <Hierarchy hasAll="true" allMemberName="All Periods" primaryKey="dateid">
        <Table name="datehierarchy"/>
        <Level name="Year" column="year" uniqueMembers="true" 
            levelType="TimeYears" type="Numeric"/>
        <Level name="Quarter" column="quarter" uniqueMembers="false" 
            levelType="TimeQuarters"/>
        <Level name="Month" column="month" uniqueMembers="false" 
            ordinalColumn="month" nameColumn="month_name" levelType="TimeMonths" 
            type="Numeric"/>
        <Level name="Week" column="week_in_month" uniqueMembers="false" 
            levelType="TimeWeeks"/>
        <Level name="Day" column="day_in_month" uniqueMembers="false" 
            ordinalColumn="day_in_month" nameColumn="day_name" 
            levelType="TimeDays" type="Numeric"/>
    </Hierarchy>
</Dimension>

Multiple hierarchies

A Dimension can contain more than one Hierarchy. By default, a Hierarchy has the same name as its Dimension.

Shared dimensions

A Dimension can contain more than one Hierarchy. By default, a Hierarchy has the same name as its Dimension.

When generating the SQL for a join, Mondrian needs to know which column to join to. If you are joining to a join, then you need to tell it which of the tables in the join that column belongs to (usually, it will be the first table in the join). Because shared dimensions don't belong to a Cube, you have to give them an explicit table (or other data source). When you use them in a particular Cube, you specify the foreign key. This example shows the Store Type Dimension being joined to the Sales Cube using sales_fact_1997.store_id as the foreignKey, and to the Warehouse Cube using warehouse.warehouse_store_id as the foreignKey:

► Example:

<Dimension name="Store Type">
    <Hierarchy hasAll="true" primaryKey="store_id">
        <Table name="store"/>
        <Level name="Store Type" column="store_type" 
        uniqueMembers="true"/>
    </Hierarchy>
</Dimension>

<Cube name="Sales">
    <Table name="sales_fact_1997"/>
    ...
    <DimensionUsage name="Store Type" source="Store Type" 
    foreignKey="store_id"/>
</Cube>

<Cube name="Warehouse">
    <Table name="warehouse"/>
    ...
    <DimensionUsage name="Store Type" source="Store Type" 
    foreignKey="warehouse_store_id"/>
</Cube>

Snowflake schema (join)

A Dimension can be based upon more than one table, provided that there is a well-defined path to join these tables to the fact table. This kind of Dimension is known as a "snowflake", and is defined using the <Join> operator.

► Example:

<Cube name="Sales">
...
    <Dimension name="Product" foreignKey="product_id">
        <Hierarchy hasAll="true" primaryKey="product_id" 
        primaryKeyTable="product">
            <Join leftKey="product_class_key" rightAlias="product_class" 
            rightKey="product_class_id">
                <Table name="product"/>
                <Join leftKey="product_type_id" rightKey="product_type_id">
                    <Table name="product_class"/>
                    <Table name="product_type"/>
                </Join>
            </Join>
            <!-- Level declarations ... -->
        </Hierarchy>
    </Dimension>
</Cube>

This defines a "Product" Dimension consisting of three tables:

• The fact table joins to the product table via foreignKey="product_id", which in turn joins to the
• product_class table via foreignKey="product_class_id", which in turn joins to the
• product_type table via foreignKey="product_type_id".

We require a <Join> element nested within a <Join> element because <Join> takes two operands; the operands can be tables, joins, or even queries.

The arrangement of the tables seems complex, but the simple rule of thumb is to order the tables by the number of rows they contain. The product table has the most rows, so it joins to the fact table and appears first. product_class has fewer rows, and product_type, at the tip of the snowflake, has fewest of all.

Note that the outer <Join> element has a rightAlias attribute. This is necessary because the right component of the join (the inner <Join> element) consists of more than one table. No leftAlias attribute is necessary in this case, because the leftKey column unambiguously comes from the product table.

Cube DSL

A Cube can be considered a multi-dimensional generalization of a two- or three-dimensional spreadsheet. Each cell of the cube holds a number that represents some measure of the business, such as sales, profits, expenses, budget and forecast. The elements of a dimension can be organized as a hierarchy, that is, a set of parent-child relationships, typically where a parent member summarizes its children.

CubeDSL generates the schema file.

The main semantic elements of the CubeDSL are:

• package - The root element that contains all the other elements. A model can contain multiple packages.
• schema - Defines the name of the Mondrian schema file
• import declarations - Used to import external models or even Java classes.
• cube - The abstraction of a Mondrian cube. It contains further elements such as dimension, measure.
• dimension - Defines an attribute, or set of attributes, which is a collection of hierarchies, and by which you can divide measures into sub-categories. For example, you might wish to break down product sales by their color, the gender of the customer and the store in which the product was sold; color, gender, and store are all dimensions.
• hierarchy - Defines a set of members organized into a structure for convenient analysis. For example, the store hierarchy consists of the store name, city, state, and nation. A hierarchy allows you to form intermediate sub-totals: the sub-total for a state is the sum of the sub-totals of all of the cities in that state, each of which is the sum of the sub-totals of all the stores in that city.
• entity - Defines which table and columns of the table the “hierarchy” uses.
• level - Defines a collection of members which have the same distance from the root of the hierarchy.
• property - Defines a member property. The definition is against a level, but the property is available to all members.
• dimensionUsage - Defines the shared dimension usage in a cube.
• measure - The members of a special dimension, a quantity being measured, for example, unit sales of a product, or cost price of inventory items.

Keywords

The CubeDSL generates a schema file in XML file type. It should be defined with the following keywords in a file with the .cube extension:

package, schema

► Syntax:

package <package name> schema <schema name>{
   ...
}

► Example:

package local.test4doku.cubes schema TEST{
   ...
}

It generates the following schema code in an XML file:

<Schema name="TEST">
   ...
</Schema>

dimension, hierarchy, level, property

► Syntax:

dimension <dimension name> {
    hierarchy [hierarchy name] [has All] [allMemberName=<all member name>]
    [defaultMember=<default member>]{
        entity <entity name> key=<key name>  {
            level <level name> column=<column name> [uniqueMembers];
            ...
        }
    }
    ...
}

Notes:

• If the hierarchy name is not defined, the default value is the dimension name.
• By default, every hierarchy contains a top level called (ALL), which contains a single member called (All {hierarchy name}). This member is the parent of all other members of the hierarchy and also the default member of the hierarchy. Use allMemberName to override the default name of the All member.
• If hasAll="false", the default member of the dimension will be the first member of the first level. Use defaultMember to override the default member of the hierarchy.

► Example:

dimension CustomerGenders {
    hierarchy allMemberName="All Genders" defaultMember="[All Genders]"{
        entity Tcustomer key=cus_id  {
            level Gender column=gender ;
        }
    }
}

It generates the following schema code in an XML file:

<Dimension name="CustomerGenders">
    <Hierarchy hasAll="false" allMemberName="All Genders" defaultMember="[All 
    Genders]" primaryKey="CUS_ID">
        <Table name="TCUSTOMER" />
        <Level name="Gender" column="GENDER" uniqueMembers="false" 
        type="String"/>
    </Hierarchy>
</Dimension>

join

► Syntax:

dimension <dimension name> {
    hierarchy [<hierarchy name>] [has All] [allMemberName=<all member name>] 
    [defaultMember=<default member>] {
        entity <entity name> key=<primarykey name>  {
            level <level name> column=<column name> [uniqueMembers];
            ...
            entity <entity name> [key=<key name>]  {
                over <leftkey name>;
                level <level name> column=<column name> [uniqueMembers];
                ...
            }
        }
    }
    ...
}

► Example:

dimension ProductTypes {
    hierarchy hasAll {
        entity Tproduct key=pro_id {
            level proName column=pro_name uniqueMembers;
                
            entity Tproduct_type key=pro_type_id {
                over type;
                level proTypeName column=pro_type_name uniqueMembers;
            }
        }
    }
}

It generates the following schema code in an XML file:

<Dimension name="ProductTypes">
    <Hierarchy hasAll="true" primaryKey="PRO_ID" primaryKeyTable="TPRODUCT">
        <Join leftKey="TYPE_ID" rightKey="PRO_TYPE_ID">
            <Table name="TPRODUCT" />
            <Table name="TPRODUCT_TYPE"/>
        </Join>
        <Level name="proTypeName" table="TPRODUCT_TYPE" 
        column="PRO_TYPE_NAME" uniqueMembers="true" type="String"/>
        <Level name="proName" table="TPRODUCT" column="PRO_NAME" 
        uniqueMembers="true" type="String"/>
    </Hierarchy>
</Dimension>

Notes:

• The default key is ID.
• For the reference to an entity, e.g., type in Tproduct_type, the name of the key in the database is automatically defined as type_id.

typeTime

► Syntax:

dimension <dimension name> typeTime {
    hierarchy <hierarchy name> [has All] [allMemberName=<all member name>] 
    [defaultMember=<default member>]{
        entity <entity name> key=<key name>  {
            level <level name> column=<column name> [uniqueMembers] 
            levelType=<type name>;
            ...
        }
    }
    ...
}

Note:

• levelType can have the values Time, TimeSeconds, TimeMinutes, TimeHours, TimeDays, TimeWeeks, TimeMonths, TimeQuarters, TimeHalfYears, TimeYears (see the definitions in the section #Time dimensions)

► Example:

dimension TimeTheDate typeTime { 
    hierarchy { 
        entity Ttime_by_day key=the_date {
            level Year column=the_year uniqueMembers levelType=TimeYears;
            level Quarter column=quarter levelType=TimeQuarters;
            level Month column=month_of_year levelType=TimeMonths;
        }
    }
}

It generates the following schema code in an XML file:

<Dimension name="TimeTheDate" type="TimeDimension">
    <Hierarchy hasAll="false" primaryKey="THE_DATE">
        <Table name="TTIME_BY_DAY" schema=""/>
        <Level name="Year" column="THE_YEAR" uniqueMembers="true" 
            levelType="TimeYears" type="String"/>
        <Level name="Quarter" column="QUARTER" uniqueMembers="false" 
            levelType="TimeQuarters" type="String"/>
        <Level name="Month" column="MONTH_OF_YEAR" uniqueMembers="false" 
            levelType="TimeMonths" type="Integer"/>
    </Hierarchy>
</Dimension>

property

► Syntax:

dimension <dimension name> {
    hierarchy <hierarchy name> [has All] [allMemberName=<all member name>] 
    [defaultMember=<default member>]{
        entity <entity name> key=<key name>  {
            level <level name> column=<column name> [uniqueMembers]{
                property <property name> column=<column name> 
                [type=<type name>];
            }
            ...
        }
    }
    ...
}

Note:

• property type can be Boolean, Date, Integer, Numeric, String, Time or TimeStamp.

► Example:

dimension ProductTypes {
    hierarchy {
        entity TproductType key= pro_type_id {
            level proTypeId column= pro_type_id uniqueMembers {
                property proTypeName column= pro_type_name type=String;
            }                
        }
    }
}

It generates the following schema code in an XML file:

<Dimension name="ProductTypes">
    <Hierarchy  hasAll="false" primaryKey="PRO_TYPE_ID">
        <Table name="TPRODUCT_TYPE" schema=""/>
        <Level name="proTypeId" column="PRO_TYPE_ID" uniqueMembers="true" 
        type="Integer">
            <Property name="proTypeName" column="PRO_TYPE_NAME" 
            type="String" />
        </Level>
    </Hierarchy>
</Dimension>

cube, dimensionUsage, measure

► Syntax:

cube <cube name> [defaultMeasure <measure name>]{
    entity <entity name> [key=<key name>]{
        dimensionUsage <dimension name> over <key name>;
        ...
        measure <measure name> <aggregator name> column=<column name> 
        [notVisible];
        ...
    }
    ...
}

Note:

• Aggregator functions include avg, sum, min, max, count, distinct-count (see the definitions in the section #Measures ).
• Mondrian takes the first measure as the default measure. You can override this behavior by setting defaultMeasure.

► Example:

dimension ProductTypes {
    hierarchy {
        entity TproductType key= pro_type_id {
            level proTypeId column= pro_type_id uniqueMembers {
                property proTypeName column= pro_type_name type=String;
            }                
        }
    }
}

It generates the following schema code in an XML file:

<Dimension name="ProductTypes">
    <Hierarchy  hasAll="false" primaryKey="PRO_TYPE_ID">
        <Table name="TPRODUCT_TYPE" schema=""/>
        <Level name="proTypeId" column="PRO_TYPE_ID" uniqueMembers="true" 
        type="Integer">
            <Property name="proTypeName" column="PRO_TYPE_NAME" 
            type="String" />
        </Level>
    </Hierarchy>
</Dimension>

Copyright Notice

All rights are reserved by Compex Systemhaus GmbH. In particular, duplications, translations, microfilming, saving and processing in electronic systems are protected by copyright. Use of this manual is only authorized with the permission of Compex Systemhaus GmbH. Infringements of the law shall be punished in accordance with civil and penal laws. We have taken utmost care in putting together texts and images. Nevertheless, the possibility of errors cannot be completely ruled out. The Figures and information in this manual are only given as approximations unless expressly indicated as binding. Amendments to the manual due to amendments to the standard software remain reserved. Please note that the latest amendments to the manual can be accessed through our helpdesk at any time. The contractually agreed regulations of the licensing and maintenance of the standard software shall apply with regard to liability for any errors in the documentation. Guarantees, particularly guarantees of quality or durability can only be assumed for the manual insofar as its quality or durability are expressly stipulated as guaranteed. If you would like to make a suggestion, the Compex Team would be very pleased to hear from you.

(c) 2016-2024 Compex Systemhaus GmbH