Showing posts with label periods. Show all posts
Showing posts with label periods. Show all posts

Monday, March 26, 2012

Grouping Data into Periods for Reporting

Hi there.

I am working on a set of reports where I am summing/averaging data elements based on what period they are in. For example, the report output should look something like this:

Period Sum May '07 41 April '07 14 Q2 '07 55 March '07 36 February '07 28 January '07 22 Q1 '07 86 June '07 N/A YTD '07 141 December '06 33 November '06 27 October '06 42 Q4 '06 102 September '06 58 August '06 84 July '06 52 Q3 '06 194 June '06 40 May '06 41 April '06 14 Q2 '06 95 March '06 67 February '06 38 January '06 N/A Q1 '06 105 YTD '06 496

For each of the items I am summing, all I have is a datetime of when the event happened. This is a relational database (not a cube), so I am struggling with how to create the 'buckets' based on period. I think the best way is to dynamically create the buckets based on a given date. Is there a way in RS that it can do this bucketing for you?

Thanks, Mike

There are several ways to "create buckets". I recently posted something here http://spacefold.com/lisa/post/Partition-Magic.aspx having discovered some of the SQL 2005 syntax that I never knew existed, which may help you in your explorations of non-cube data -- and there is also a PIVOT clause which is really neat.

When you look at it, it looks as though you can't dynamically figure out how many buckets you have and go for it, but you actually can, if you write a bit of very easy dynamic SQL. I was actually planning on posting about that today, having helped a co-worker do it!

But in RS, you can do this using a matrix layout control, which basically does the thing for you, albeit with (from my POV) some frustrating and counter-intuitive ways of thinking.

Look into the matrix data region first, if you need to display the buckets across, and if you don't like it look into the PIVOT clause to do this in SQL Server (NB: if your data source isn't SQL Server you don't have this T-SQL syntax but there are ways of getting around that if you need to <g>.)

If you need to display the buckets down, I think you have even easier ways to do it. What you seem to be showing (as I read your example table) is a table that has grouping and you've suppressed the detail rows that might ordinarily appear with each line (the dates for each event). OK so far?

The very simplest way to get what you want is to write your query like this:

SELECT MONTH(eventdate) AS M, Datepart(quarter,eventdate) AS Q, YEAR(eventdate) AS Y, eventtype, eventdate from YourEventTable

... now you can summarize by having appropriate groups on the first three calculated values that you see in this query.

The wrinkle that most people have when doing this particular thing is that they actually want the fiscal month, the fiscal quarter, and the fiscal year rather than the base values that you see here. So you generally want to write a couple of UDFs that pass in your first month of fiscal year to handle this properly. These can be a PITA but once you've written them appropriately for your situation, you are usually okay forever. Give a shout if you find you need help with this part. (I may be offline for about a week, but if it is urgent I'm sure many other people can help you with this).

So that's how you do it if your "buckets" are rows, as they seem to be from your example layout. If they are really columns, again, look into matrix and pivot.

>L<

|||

Hi Lisa.

Thanks so much for the great response. I was working on doing something similar, but there are a few more wrinkles (aren't there always?). As you mentioned, the buckets are rows, so that is good, but in this case, my columns can vary, so I need to use a matrix control.

1. The data set I need is for the current quarter (based on getdate()) and the previous 5 full quarters. I think this can be easily handled in the where clause, so that should be ok.

2. The matrix control is strange in that when you add row groups, it actually adds them as columns on the report itself (I posted a question on this recently). My customer doesn't want it that way, so I have to get the dataset to match the control - meaning, I will need to have my dataset return the summed/averaged bucket rows and NOT have the control handle the buckets. So, I think I am stuck.

Can you see any way around that?

Thanks, Mike

|||

Hi again, Mike,

I can't actually think closely about what you're stuck on here, because (I think I said) I'm leaving on a trip and 'way late on preparing <s>. But, in fact, a PIVOT clause might be just the ticket here -- for this reason among others I did end up posting a blog entry about that. http://spacefold.com/lisa/post/Matrix-Rebuilt-More-non-standard-fun-with-T-SQL.aspx

It's discussing some aspects of the clause that you may or may not be interested it but it will give you some idea of the scope of what it can help you accomplish. In your case -- since you actually know how many buckets there are (6 quarters) -- it may be especially apt.

I'll be back in a week, if you haven't got what you need by then I'll do my best to help <s>. Look forward to reading whatever else has been posted on this thread by then!

>L<

|||

Thanks Lisa.

I'll check into it. Enjoy your time 'away'.

- Mike

Friday, February 24, 2012

Group By

I'm trying to aggregate values and grouping the results by defined periods (6
month, 12 month, 18 month, and lifetime). I'd like to do this with one pass,
but using a CASE statement did not aggregrate correctly. The results are
more like 6 months, 7-12 months, and 13-18 months, 19-lifetime. I would like
the results of the 12 month grouping to include all of the 6 month, the 18
month to include all of the 6 and 12 month grouping, etc.
/**/
CASE WHEN pd.accountingdate > 0 THEN 'LIFE'
WHEN pd.accountingdate BETWEEN '10/19/2004' AND '04/19/2007' THEN '18MONTH'
WHEN pd.accountingdate BETWEEN '04/19/2006' AND '04/19/2007' THEN '12MONTH'
WHEN pd.accountingdate BETWEEN '10/19/2006' AND '04/19/2007' THEN '06MONTH'
ELSE NULL
END
/**/
Thanks.
"Jeremy" <Jeremy@.discussions.microsoft.com> wrote in message
news:79FBE5E0-E112-406A-B406-A5D19626FA96@.microsoft.com...
> I'm trying to aggregate values and grouping the results by defined periods
> (6
> month, 12 month, 18 month, and lifetime). I'd like to do this with one
> pass,
> but using a CASE statement did not aggregrate correctly. The results are
> more like 6 months, 7-12 months, and 13-18 months, 19-lifetime. I would
> like
> the results of the 12 month grouping to include all of the 6 month, the 18
> month to include all of the 6 and 12 month grouping, etc.
>
> /**/
> CASE WHEN pd.accountingdate > 0 THEN 'LIFE'
> WHEN pd.accountingdate BETWEEN '10/19/2004' AND '04/19/2007' THEN
> '18MONTH'
> WHEN pd.accountingdate BETWEEN '04/19/2006' AND '04/19/2007' THEN
> '12MONTH'
> WHEN pd.accountingdate BETWEEN '10/19/2006' AND '04/19/2007' THEN
> '06MONTH'
> ELSE NULL
> END
In situations like this, you should post DDL, sample data, and the actual
query. A non-working snippet of a query doesn't really help anyone
understand the complete situation.
Making some assumptions, I believe your problem is related to the use of a
single column to represent different periods. The above case expression
represents what? Is it the 6 month data, the 12 month data, ...? It can't
represent more than one "attribute" - in this case, period.
The solution is to generate separate period values. This can be done in one
of two ways. Either you aggregate the periods as separate columns or you
create a situation where you join the data to be aggregated to a table
containing the periods. In the first example, you get period data as
separate columns within the result set. In the second example, you get
period data as separate rows. You decide which way you want to proceed. It
will facilitate discussion to use either Pubs or Northwind for sample data
and queries since most people have those available (and thus do not require
the posting of DDL or sample data).
|||Understood Scott, I've already considered the approach you've suggested, but
let me expand to help everyone better understand (as you recommended).
Using a case statement gives me the following from the Orders table in
Northwind:
customerid|period|orders|freight
AROUT|06MONTH|8|275.6900
AROUT|12MONTH|2|94.7100
AROUT|18MONTH|3|101.5500
BERGS|06MONTH|7|612.0700
BERGS|12MONTH|5|419.5500
BERGS|18MONTH|4|426.2300
BERGS|LIFE|2|101.6700
/* sample statement */
SELECTcustomerid,
CASEWHEN orderdate BETWEEN '11/06/1997' AND '05/06/1998' THEN '06MONTH'
WHEN orderdate BETWEEN '05/06/1997' AND '05/06/1998' THEN '12MONTH'
WHEN orderdate BETWEEN '11/06/1996' AND '05/06/1998' THEN '18MONTH'
WHEN orderdate > 0 THEN 'LIFE'
ELSE NULL
END period,
COUNT(orderid) orders,
SUM(freight) freight
FROMorders
WHEREcustomerid IN ('AROUT', 'BERGS')
GROUP BY
customerid,
CASE WHEN orderdate BETWEEN '11/06/1997' AND '05/06/1998' THEN '06MONTH'
WHEN orderdate BETWEEN '05/06/1997' AND '05/06/1998' THEN '12MONTH'
WHEN orderdate BETWEEN '11/06/1996' AND '05/06/1998' THEN '18MONTH'
WHEN orderdate > 0 THEN 'LIFE'
ELSE NULL
END
/**/
However to get the results I seek I have to make four passes:
customerid|period|orders|freight
AROUT|06MONTH|8|275.6900
AROUT|12MONTH|8|275.6900
AROUT|18MONTH|13|471.9500
AROUT|LIFE|13|471.9500
BERGS|06MONTH|7|612.0700
BERGS|12MONTH|7|612.0700
BERGS|18MONTH|16|1457.8500
BERGS|LIFE|18|1559.5200
/* sample statement */
SELECTcustomerid,
'06MONTH' period,
COUNT(orderid) orders,
SUM(freight) freight
FROMorders
WHEREcustomerid IN ('AROUT', 'BERGS')
AND orderdate BETWEEN '11/06/1997' AND '05/06/1998'
GROUP BY customerid
UNION
SELECTcustomerid,
'12MONTH' period,
COUNT(orderid) orders,
SUM(freight) freight
FROMorders
WHEREcustomerid IN ('AROUT', 'BERGS')
AND orderdate BETWEEN '11/06/1997' AND '05/06/1998'
GROUP BY customerid
UNION
SELECTcustomerid,
'18MONTH' period,
COUNT(orderid) orders,
SUM(freight) freight
FROMorders
WHEREcustomerid IN ('AROUT', 'BERGS')
AND orderdate BETWEEN '11/06/1996' AND '05/06/1998'
GROUP BY customerid
UNION
SELECTcustomerid,
'LIFE' period,
COUNT(orderid) orders,
SUM(freight) freight
FROMorders
WHEREcustomerid IN ('AROUT', 'BERGS')
AND orderdate > 0
GROUP BY customerid
ORDER BY customerid, period
/**/
The results of the first query are affected by the sequence of the
conditions in the CASE statement. And there's the rub. I'd like to have my
results in rows because I'm aggregrating at least 20 columns and 4 periods
would push it to 80 columns. Thanks for you help.
"Scott Morris" wrote:

> "Jeremy" <Jeremy@.discussions.microsoft.com> wrote in message
> news:79FBE5E0-E112-406A-B406-A5D19626FA96@.microsoft.com...
> In situations like this, you should post DDL, sample data, and the actual
> query. A non-working snippet of a query doesn't really help anyone
> understand the complete situation.
> Making some assumptions, I believe your problem is related to the use of a
> single column to represent different periods. The above case expression
> represents what? Is it the 6 month data, the 12 month data, ...? It can't
> represent more than one "attribute" - in this case, period.
> The solution is to generate separate period values. This can be done in one
> of two ways. Either you aggregate the periods as separate columns or you
> create a situation where you join the data to be aggregated to a table
> containing the periods. In the first example, you get period data as
> separate columns within the result set. In the second example, you get
> period data as separate rows. You decide which way you want to proceed. It
> will facilitate discussion to use either Pubs or Northwind for sample data
> and queries since most people have those available (and thus do not require
> the posting of DDL or sample data).
>
>
|||Understood Scott. I've already considered the approaches you've suggested,
but let me expand to help us better understand my problem as you recommended.
The CASE statement approach gives me the following results from the Orders
table in Northwind:
customerid|period|orders|freight
AROUT|06MONTH|8|275.6900
AROUT|12MONTH|2|94.7100 /* freight should = 370.40*/
AROUT|18MONTH|3|101.5500
BERGS|06MONTH|7|612.0700
BERGS|12MONTH|5|419.5500
BERGS|18MONTH|4|426.2300
BERGS|LIFE|2|101.6700
/* sample code */
SELECTcustomerid,
CASE
WHEN orderdate BETWEEN '11/06/1997' AND '05/06/1998' THEN '06MONTH'
WHEN orderdate BETWEEN '05/06/1997' AND '05/06/1998' THEN '12MONTH'
WHEN orderdate BETWEEN '11/06/1996' AND '05/06/1998' THEN '18MONTH'
WHEN orderdate > 0 THEN 'LIFE'
ELSE NULL
END period,
COUNT(orderid) orders,
SUM(freight) freight
FROMorders
WHEREcustomerid IN ('AROUT', 'BERGS')
GROUP BY
customerid,
CASE
WHEN orderdate BETWEEN '11/06/1997' AND '05/06/1998' THEN '06MONTH'
WHEN orderdate BETWEEN '05/06/1997' AND '05/06/1998' THEN '12MONTH'
WHEN orderdate BETWEEN '11/06/1996' AND '05/06/1998' THEN '18MONTH'
WHEN orderdate > 0 THEN 'LIFE'
ELSE NULL
END
/**/
However, I would like my results to read:
customerid|period|orders|freight
AROUT|06MONTH|8|275.6900
AROUT|12MONTH|10|370.4000
AROUT|18MONTH|13|471.9500
AROUT|LIFE|13|471.9500
BERGS|06MONTH|7|612.0700
BERGS|12MONTH|12|1031.6200
BERGS|18MONTH|16|1457.8500
BERGS|LIFE|18|1559.5200
/* sample code */
SELECTcustomerid,
'06MONTH' period,
COUNT(orderid) orders,
SUM(freight) freight
FROMorders
WHEREcustomerid IN ('AROUT', 'BERGS')
AND orderdate BETWEEN '11/06/1997' AND '05/06/1998'
GROUP BY customerid
UNION
SELECTcustomerid,
'12MONTH' period,
COUNT(orderid) orders,
SUM(freight) freight
FROMorders
WHEREcustomerid IN ('AROUT', 'BERGS')
AND orderdate BETWEEN '05/06/1997' AND '05/06/1998'
GROUP BY customerid
UNION
SELECTcustomerid,
'18MONTH' period,
COUNT(orderid) orders,
SUM(freight) freight
FROMorders
WHEREcustomerid IN ('AROUT', 'BERGS')
AND orderdate BETWEEN '11/06/1996' AND '05/06/1998'
GROUP BY customerid
UNION
SELECTcustomerid,
'LIFE' period,
COUNT(orderid) orders,
SUM(freight) freight
FROMorders
WHEREcustomerid IN ('AROUT', 'BERGS')
AND orderdate > 0
GROUP BY customerid
ORDER BY customerid, period
/**/
The results of the first query are affected by the sequence of the
conditions in the CASE statement. And there's the rub. I need the 18 month
period to include 6 months and 12 months. The only way I know to do that is
through multiple passes. I prefer my results to rows as I'm aggregating at
least 20 columns and if I push the results to columns with 4 different
periods, then I would expect at least 80 columns! Any help is greatly
appreciated. Thanks.
"Scott Morris" wrote:

> "Jeremy" <Jeremy@.discussions.microsoft.com> wrote in message
> news:79FBE5E0-E112-406A-B406-A5D19626FA96@.microsoft.com...
> In situations like this, you should post DDL, sample data, and the actual
> query. A non-working snippet of a query doesn't really help anyone
> understand the complete situation.
> Making some assumptions, I believe your problem is related to the use of a
> single column to represent different periods. The above case expression
> represents what? Is it the 6 month data, the 12 month data, ...? It can't
> represent more than one "attribute" - in this case, period.
> The solution is to generate separate period values. This can be done in one
> of two ways. Either you aggregate the periods as separate columns or you
> create a situation where you join the data to be aggregated to a table
> containing the periods. In the first example, you get period data as
> separate columns within the result set. In the second example, you get
> period data as separate rows. You decide which way you want to proceed. It
> will facilitate discussion to use either Pubs or Northwind for sample data
> and queries since most people have those available (and thus do not require
> the posting of DDL or sample data).
>
>
|||> The results of the first query are affected by the sequence of the
> conditions in the CASE statement. And there's the rub. I'd like to have
> my
> results in rows because I'm aggregrating at least 20 columns and 4 periods
> would push it to 80 columns. Thanks for you help.
The secret is that you need a table of periods - which you then join to your
data while aggregating. This can be done in one pass since each "raw data
row" joins once to each appropriate period. So a row for today joins to the
6 month period while a row from 10 months ago joins to both the 6 and 12
month periods (and so forth). The trick is to generate the period table -
dynamically if needed. Below is a query (that can be used as a derived
table) to do this. Obviously, the variable isn't actually required. You
could create a table-valued function to do the same thing (especially if you
needed to reuse this logic).
declare @.today datetime
set @.today = DATEDIFF(DAY, 0, CURRENT_TIMESTAMP)
select @.today, dateadd(month, -6, @.today)
union all
select @.today, dateadd(month, -12, @.today)
union all
select @.today, dateadd(month, -18, @.today)
You may need to adjust the logic depending on your definition of periods -
the last day of the month (esp Feb in and out of a leap year) is always fun
for these types of things. With the table of periods, you simply join to
the transaction table where date of transaction is between the period start
and end dates.
|||Your solution is perfect! This increases performance and keeps my DBA happy.
I'll add this technique to my bag of tricks. Thanks Scott!
"Scott Morris" wrote:

> The secret is that you need a table of periods - which you then join to your
> data while aggregating. This can be done in one pass since each "raw data
> row" joins once to each appropriate period. So a row for today joins to the
> 6 month period while a row from 10 months ago joins to both the 6 and 12
> month periods (and so forth). The trick is to generate the period table -
> dynamically if needed. Below is a query (that can be used as a derived
> table) to do this. Obviously, the variable isn't actually required. You
> could create a table-valued function to do the same thing (especially if you
> needed to reuse this logic).
> declare @.today datetime
> set @.today = DATEDIFF(DAY, 0, CURRENT_TIMESTAMP)
> select @.today, dateadd(month, -6, @.today)
> union all
> select @.today, dateadd(month, -12, @.today)
> union all
> select @.today, dateadd(month, -18, @.today)
> You may need to adjust the logic depending on your definition of periods -
> the last day of the month (esp Feb in and out of a leap year) is always fun
> for these types of things. With the table of periods, you simply join to
> the transaction table where date of transaction is between the period start
> and end dates.
>
>