Dwh lecture 13-process dm

Editor's Notes

• #3 The four steps are: but before the difference b/w er and dm should be clear. Er covers the whole business, which has number of processes like accounting, inventory, invoices or the process which do not work simultaneously. But as er by defination covers the whole business so it covers the all processes. The complexity of er process understanding is difficult. A process could be in case of exceptional conditions what should be done. But in dm we isolates the process, and this is one reason to know the dm. it identifies the processes, and then dm is developed for that process. The second point is to choose the grain of the process. Grain is high then detail will be high as well. In fact table more records or details will come in this case. The grain level b/w sys and fact table could be different (detail later) Now the third step is to check the facts or isolate them. Facts are numeric value but it shouldn’t be the key value. So fact is a non key value. So by choosing facts you made fact tables. Now the dimension tables are great in number and small in size, and you choose them. They are joined with fact table, you have to identify them.
• #4 It is not an automated process could be done by a wizard, which needs domain knowledge and human experience which needs efforts. Business process is the major process runing in a business. These are the core business processes which are controlled by oltp systems.
• #5 This is an animation. In this snowflake schema and it has a great number of processes which are represented as star which needs the dimensional modeling. The system which is developed against these stars are known as data marts.
• #6 Grain is atomic level means which couldn’t be breaked. Grain corresponds to a transaction, which goes in the table. The detail level of data which is captured in business, that level and the level of data which goes in fact table may be different. E.g. in a store a trasaction could be of single item, or multiple items. It could be on the base of sales per hour, week etc. When we talk about grain with transaction, it has a factor of exprssivness. The more detail, means you know more about your business. Less data less expressive. So if the aggregation is high you will not need to invest on hardware for the performance. So the focus should be on grain, i.e if you want to run the queries in parallel you need more processing power. But high aggregation means when you need answer you will not need to run the query in parallel you just need to display the precomputed aggregate against the query.
• #7 Grain has 2 costs. 1 grain making needs effort. E.g. you need to down the dwh, so for grain making, you need the time other than the office hours i.e 2 am. So you need support people so you need dollars. After that when you will make the grains, you need to store them. So benifitial when you need to use again and again. So the queries should be which may use the grain again and again. Aggregates are business process dependent. When you make aggregates you keep the business before you. So they are more benifitial.
• #8 If I take the weekly aggregates I can make monthly aggregates, but if aggregates are monthly can you come back to weekly aggregates. E.g we can guess 3+5 but can not tell if 8 is given and you want to know 2 numbers. As already discuss several “w” are involved in aggregation, when you aggregate you remove one w. So suppose you aggregate on geography you remove where “w”. So now you cannot tell, this sale is either of zone1,2 etc. suppose you summarized on geography you cannot tell what actually happened in zones as geography has zones as components. The reason is aggregation is irreversible and from aggregation we mean summary. so if you aggregate on when i.e time you cannot tell when that event occurred. Aggregation is one-way i.e. you can create aggregates, but can not dissolve aggregates to get the original data from which the aggregates were created. For example 3+2+1 = 6 at the same time 2+4 also equals 6, so does 5+1 and if we consider reals, then infinetly many ways of adding numbers to get the same result. If you think about the “5 W’s of journalism”, these are the “6 W’s of data analysis”. Again it highlights the types of questions that end users want to ask and can not be answered by summary data. By definition, a summarization will consider at least one of these points irrelevant. For example, a summary across the company takes out the dimension of “WHERE” and a summary by quarters takes out the element of “WHEN”. The point to be noted is that although summary data has a purpose, yet one can take any summary and ask a question that the system can not answer.
• #9 This is not the case of ireversibility, it means same aggregate can come by adding different numbers. But a lot of pit falls can come.
• #10 In this example a portion of data is hide, here it represents that the average of every zone in every week. Now from aggregation, we can see that average sale is equal in all zones. Suppose a company has launched a sale compaign, and they want to know the impact of the compaign in all zones. Now aggregation has hide the information, and if we look at its graph in next sl
• #12 Questioon is what are facts and dimensions. If in this statement when we say we need sales volume and Rs these are facts. And dimension mantioned. So it is your job to identify what are the facts and what are the dimensions.
• #13 Every number is not fact i.e. pk. The rule is you suld be able to add them, e.g. discount is not fact, because you give discount on the base of percentage (discuss later).
• #14 Dimensions are descriptions, which have hirarchy or you run where clause on it. You have to identify the dimensions you cannot judge them from their loud voices.
• #15 Its mean when a transaction is recorded, during transaction the attribute which do not change its value is dimension. See the above table. Here, you went to an atm and you withdraw some money, now transaction_RS attribute will not have that money which was previously their, but there are some attributes like location, account_no they will not change. Now question is time changes but here we are concerned with time_of_day like is it morning, evening etc. so the time will remain constant because transaction will not take so long. So except transaction_rupees every thing is dimension.
• #16 Are dimensions Always single valued: a car selling company has a workshop as well. So when car goes there they record different type of data against it. Oil filter change, plug change so maintainance has multivalues. So multi values are possible in dimension table.
• #17 Several grains are possible but select the most appropriate grain low grains will have more dimensions ie you aggregate on day, then time of day will have several values like morning, evening etc. but if the grain is weekly then it wont have such dimensions. So low grains are more expressive.