Made2Mentor » SSRS

Stop Creating Visual FoxPro Reports… Today

David — Tue, 05 Jul 2011 14:10:34 +0000

Last week I saw a discussion on the LinkedIn Made2Manage Group about the Pros and Cons of using SSRS vs. VFP Reporting, and it amazes me that people are still arguing about it. So, I thought I’d reaffirm a stance that I’ve had for many years, which is VFP Reporting Must Die.

I’ve lamented Visual FoxPro’s limitations in the past and I don’t want bore you by hashing them all out again. Let’s just hit the highlights.

VFP reports are difficult to write and maintain.
VFP lacks chart and graph capability.
M2M report permissions management is tedious.

However, in addition to that list, there are many other reasons you should curtail your use of Visual FoxPro.

Technological Advantages

There many technical advantages to using SSRS (or even Crystal Reports for that matter) over Visual FoxPro. In addition to graphing capabilities, SSRS also facilitates more interaction and customization. For example, you can easily create reports with drill down, and you can drill across to different reports as well. Let’s say for example, that your Operations Manager is looking at a Sales Order report, and wants to drill across to Purchase Order information tied to that Sales Order. Think you can do that with VFP?

Develop Marketable Skills

As I mentioned last week, VFP is a dying language and will never make you a more desirable employee in the market. The same applies if M2M adopts some other niche product as their new reporting strategy. Nobody is going to pay six figures for experience in some product nobody uses. Spend time focusing on T-SQL Programming and SSRS instead.

Remember, that it’s in Consona’s best interest to use a proprietary niche product as that generates additional consulting revenue for them.

Ease of Use

SSRS is so much easier to use than Visual FoxPro. It’s easier to learn and the tools are more polished.

Re-usable Code

All of these methods essentially boil down to one common denominator, T-SQL. The base T-SQL code is more easily extracted from SSRS, Crystal, etc. than from Visual FoxPro. Also, once you have the T-SQL, you can quickly re-create reports in any of the other methods including Qlickview, Tableau, and Excel.

Also, don’t forget to use Database Views to make your code more manageable and easier to upgrade.

SSRS is Less Expensive Than VFP

SSRS is free with SQL Server 2005 and above. Did you actually license all of those copies of Visual FoxPro that you most likely have on your server and several workstations? If not, you’re breaking the law as those licenses cost approximately $500 a pop.

SSRS Consulting is Less Expensive

M2M charges $225 an hour for report customization work. One of the reasons they can is because there are relatively few people who are skilled in Visual FoxPro Reporting. SSRS Reporting skills are much more common and the competition drives prices down. Competent SSRS report writers can be hired for anywhere from $100 – $150 per hour on contract.

The Made2Manage people who recommend VFP Report Writing have three basic reasons that they cite for doing so.

Using Anything Else is Not Supported

Yes, this is true. Consona does not support the use of SSRS, Crystal, or even Excel for that matter. However, they don’t support their own VFP report customizations either. Consona’s contracts generally state that the contracted customization is only guaranteed to work on the version for which it was written. Be prepared, no matter which reporting method you use, to test and possibly edit your custom reports when migrating to a new version of M2M. If you’re using M2M Customized VFP reports, that may well mean more billing hours at $255 per.

M2M Version 7 is Imminent

Really? M2M has just begun to ask for Beta Testing volunteers. That process, especially when migrating to a new major release, will take a long time. And really, even if Version 7 was available today, how soon will most of you migrate anyway? I personally wouldn’t move until 7.01 (or after several service packs were issued) as new major releases are always problematic. Just ask the poor souls who were early adopters of version 6.0.

Yes, in a future version, M2M will get away from using Visual FoxPro. However, to the best of my knowledge they still haven’t announced what their new method will be. Even if it is some proprietary product, does it really matter anyway? They certainly aren’t going to migrate the entire product to Oracle or another database platform. Your SSRS reports will still work, and the time and effort invested in them is not wasted.

Opening Reports in Another Program is Inconvenient

You don’t need to open SSRS to use the reports. Those of you who regularly read this blog know that I’ve already shared how you can launch SSRS reports quickly and easily directly from M2M. I know that people have done the same thing with Crystal Reports as well.

One Exception….

The only time I edit a M2M VFP Report is when the requested customization is very small and easy to deliver. If a user wants another field from a table in the query, or some other small task, I’ll edit the current VFP report instead. I only do that when it’s more efficient than creating an entire report in SSRS or another medium.

What do you think? Anyone disagree with what I’ve said? What are you using as your reporting method and why?

Opening SSRS Reports Automatically From Other Systems

David — Fri, 27 May 2011 14:43:06 +0000

Users typically love SQL Server Reporting Services Reports (SSRS), especially if they’re accustomed to more primitive tools such as Visual FoxPro. However, One of the biggest complaints I hear about SSRS is that users want to be able to launch it directly from their ERP, accounting, and other systems. A conventional way to do this is through URL (Universal Resource Locator) manipulation.

Depending on the capabilities of the source system, reports can be launch from a button, combo box, list box, etc. and display a specific recordset based upon parameters you specify. The best way I’ve found to launch SSRS from Made2Manage is through VBA, which is a free optional module that any M2M customer can use.

Steps to Create a URL Link Customization

Create a basic SSRS report. For this demo, I’ve created a very simple report which pulls a set of records from the Sales Order Master (SOMast) Table, which will be launched from the Sales Screen (SO).

Determine the actual URL link. I’m not going to go through each step of obtaining the proper URL because there’s an excellent step by step tutorial here. In my case, the URL will be:

http://jeditemple/ReportServer/Pages/ReportViewer.aspx%2fBlog+Projects%2fSales+Details+for+URL+Link&rs:Command=Render

Use the tutorial link instructions to find your URL, and the rest of this article is about how to integrate this into M2M.

Properly install VBA on the systems which will use this link. There are instructions on M2MExpert as well as on the M2M Install Disks, but beware that they are confusing and hard to follow.
Customize your M2VEvents.prj file in VBA. There is a downloadable class on M2M’s website for VBA programming.
You manipulate your M2VEvents.prj file by selecting Tools, Macros, and then Macros or by clicking on the Transfer button and typing VBPROJ. If your VBA project file is empty, you will have to access it by opening a screen in M2M, in this case SO, right clicking on the screen mnemonic, and selecting components. Choose INIT and the VBA Editor will open and the SO module will be created for you.

Right click on the Module folder and select Insert and Module.

Click on the new module and change the name to “DefaultModule” in the properties box.

Open the default module and paste the following code into it. This is a public function that is called by each module when creating the report URLs. Notice that I’ve ended it after the Blog Projects folder. My test SSRS Server only has the one folder on it at this time. The point is that this function returns the base portion of the URL for every report you intend to launch from M2M. The reason I’ve used the DefaultModule is if I decide to commission a new server, with a new name and such, I only need to change this base portion in one place, not in every module of the VBA project. Keep in mind that you will need to customize this code to match your server, folder structure, report names, etc.

Public Function GetReportString() As String
    GetReportString = "http://jeditemple/ReportServer/Pages/ReportViewer.aspx?%2fBlog+Projects%2f"
End Function

Open the SO Module, press CTRL-A to select all of the text in it and delete that text. Then copy and paste in the following code into the SO module.

Public ReportStr As String
Public Function INIT() As Boolean
 
 
   With ThisForm.m2mPageFrame1.page1
     .addobject "lstReports", "m2mListBox"
     .lstreports.Top = 300
     .lstreports.Left = 500
     .lstreports.Width = 80
     .lstreports.Height = 39
     .lstreports.Visible = True
     .lstreports.AddItem "Sales"
    End With
 
   ReportStr = GetReportString()
 
    INIT = True
End Function
 
Public Function M2mpageframe1_Page1_lstReports_CLICK() As Boolean
 
Dim Rpt As String, Param As String, lcSomastSono As String, Newsite As Object
 
Set Newsite = CreateObject("InternetExplorer.application")
 
Select Case ThisForm.m2mPageFrame1.page1.lstreports.Value
 
    Case "Sales"
            lcSomastSono = Trim(ThisForm.gettablevalue("somast", "fsono"))
            Rpt = "Sales+Details+for+URL+Link&rs:Command=Render"
            Param = "&SONO=" + lcSomastSono
            Newsite.Navigate (ReportStr & Rpt & Param)
            Newsite.Visible = True
    Case Else
            MsgBox ("Problem occured. Contact your administrator for help. ")
End Select
    M2mpageframe1_Page1_lstReports_LICK = True
End Function

The INIT code draws a listbox on the SO form, assigns it’s properties, adds a selection to it called “Sales”. I chose a listbox because it’s one of the few controls on an M2M form that isn’t constrained by the Edit State of the screen. In other words, you can click on it even if you’re just viewing orders rather than editing them. I’d like to find a way to create something that looks like a hyperlink, but have not yet been successful.
The second function, which fires when the listbox is clicked, creates the URL from it’s basic parts.

ReportStr as stated before is the base of your URL including your server name and base folder.
Rpt is the URL portion with the name of your report.
Param is the name of your parameter for this specific report.
lcSomastSono contains the record value, in this case the current Sales Order Number.

If you’ve coded it properly, when you open the SO screen in M2M, you should see the following:

If you’ve modified the code properly to connect to your SSRS server, then the report should open up to the corresponding sales order. In my case the URL created is shown below. Notice that I’ve bolded the custom parameter portion.

http://jeditemple/ReportServer/Pages/ReportViewer.aspx?%2fBlog+Projects%2fSales+Details+for+URL+Link&rs:Command=Render&SONO=000064

There are additional parameter options shown in that tutorial which allow you to suppress the report toolbar, the parameter portion, etc.

Although the code may look cumbersome, once you work through it, it’s very easy to understand and replicate. The VBA Modules are available for download as well.

Any questions or suggestions for improvement?

Drawing a Blank. Dealing With Blank Values in SQL Server using NullIf and Coalesce

David — Thu, 17 Mar 2011 15:08:18 +0000

“Melodical Murray is a human musical instrument; he makes music by blowing his…blank” – Match Game

In the previous article I discussed the issues when working with Null-able fields in Joins and Where clauses. This article will deal with the problems of blank values.

What About Blanks?

Many systems forego the use of Null-able fields, but instead use blank values. In fact, blank values are a bigger problem in M2M than Nulls. A common problem is with M2M Product Class values. For those that don’t use M2M, Product Class is similar to a Product Category that happens to be tied to general ledger accounts. Unfortunately, not every record in sales and other tables has a Product Class value and the value is blank. If you weren’t aware of this, run the following query against your M2M Database.

SELECT DISTINCT ARI.fprodcl ProductClass
FROM   dbo.aritem ARI

The M2M Educational Database returns the following:

In order to demonstrate this for those who don’t use M2M, I’ll run the following against the Kung Fu Database.

UPDATE dbo.OrderDetails
SET    ItemDescription = ''
WHERE  orderid = 4
 
SELECT DISTINCT OD.ItemDescription
FROM dbo.OrderDetails OD

So, how can we deal with the blank value (and the Null value as well?) Well, we could use a Case statement.

 SELECT DISTINCT
       CASE
         WHEN OD.ItemDescription IS NULL THEN '..Null None'
         WHEN OD.ItemDescription = '' THEN '..Blank None'
         ELSE OD.ItemDescription
       END
         AS ItemDescription
  FROM dbo.OrderDetails OD

Noticed that I assigned the value to Null and blank separately so you could see that it worked. Why did I use two periods before the text? I’ll explain that in a minute. The Case When works, but it’s wordy. There’s a much easier way to deal with both Blanks and Nulls.

NullIf and Coalesce

Let’s start with the NULLIF function. NULLIF accepts two arguments and if the arguments are the same, then Null is returned, otherwise the first value is returned. Therefore, we can detect blanks with the following:

 
SELECT NULLIF ('A', '')
 -- Returns 'A'
SELECT NULLIF ('', '') 
-- Returns Null
Declare @Test Char(5)
SELECT NULLIF (@Test, '') 
-- Returns Null
-- I had to use a variable for the last test because the first argument must 
-- have a known data type or an error will result.

The second argument in this case is always ” (blank). Why? If the field value is blank, the end result will be Null. Also, if the first argument is Null, that will return as Null as well. Make sense?

A Coalesce statement can then be used to replace the resulting Null value like so:

SELECT COALESCE(NULLIF ('A', '') , '..None') 
-- Returns 'A'
SELECT COALESCE(NULLIF ('', '') , '..None') 
--Returns '..None' 
Declare @Test Char(5)
SELECT  COALESCE(NULLIF (@Test, '') , '..None') 
-- Returns '..None'

Let’s return to our original statement and substitute this method.

SELECT DISTINCT
       coalesce (nullif (OD.ItemDescription, ''), '..None')
         AS ItemDescription
  FROM dbo.OrderDetails OD

I have this process programmed as a Toad Toad for SQL Server snippet. When I have time I’ll record a video showing how to use them. In this case, all I have to do is highlight the database field name, click the snippet, and Toad does the rest. So, why do I substitute text with leading periods? I do that primarily for generating available values in SSRS parameters.

When I go back and edit the first script (for M2M Product Classes) with this method, I get the following:

SELECT DISTINCT
       ARI.fprodcl ProductClass
      ,coalesce (nullif (ARI.fprodcl, ''), '..None') FixedProductClass
  FROM dbo.aritem ARI
ORDER BY coalesce (nullif (ARI.fprodcl, ''), '..None')

Which would your users rather see in a SSRS parameter drop down?

It’s important to note that blank (”) is an actual value, which is in fact equal to another blank, and can be used to join as well. However, in the case of Made2Manage, you shouldn’t expect to join your tables to the Product Class Table (INProd) with an Inner Join because you’ll “lose” records. That’s why I always link to INProd with a Left Outer Join. Would you like to see the invoice records that you’d lose in an Inner Join? Run the following against your database to find out.

SELECT ARM.fbcompany AS BillToCompany
      ,ARI.fcinvoice AS InvoiceNo
      ,ARI.fitem AS ItemNo
      ,ARI.fprodcl AS ProductClass
      ,rtrim (ARI.fpartno) AS PartNo
      ,ARI.frev AS PartRev
      ,ARI.ftotprice AS NetExtInvoiced
  FROM   dbo.armast ARM
       INNER JOIN
         dbo.aritem ARI
       ON ARM.FCINVOICE = ARI.FCINVOICE
 WHERE ARM.fcstatus <> 'V' AND
 
       coalesce (nullif (ARI.fprodcl, ''), '..None') = '..None' -- Returns Blanks and Nulls

In the M2M Educational Database, 19 records are returned.

So, in these two articles, I’ve demonstrated the problems of dealing with Nulls and Blanks in your database and how to deal with them. In the future, I’ll follow up with articles on creating and using Date tables and other more advanced T-SQL techniques.

Stay tuned!

I Don’t Know! Deal With It! (Nulls)

David — Tue, 15 Mar 2011 15:59:40 +0000

This topic has been done to death in many places, but I still get frequent questions about Null field issues in T-SQL. My purpose in this article is not to debate the definition of Null. Smarter men than I have already done so.

SQL Server Books Online indicates “A value of NULL indicates that the value is unknown.” However, some disagree, most recently Joe Webb – Null is not Unknown. Some argue that there should be several different types of Null.

Why Ask Why?

As I said before, I don’t really care about the various interpretations of the Null value. I just know that they’re a pain to deal with. I have never received questions like this:

What exactly is a Null?
What’s the definition of Null?

The questions I get follow along these lines.

Why am I losing records in this join?
Why is my total (or average) off in this query?
Why does my SSRS parameter list look funny?
Why won’t my parameters default like they should?

So dear reader, as part of the T-SQL Basics Series, I am going to share what you need to know about dealing with Nulls rather than defining them. If you’d like to follow along, please download the Kung Fu Database Script.

Where Clause

To start with, while people disagree about the definition of Null, everyone agrees what Null is not. It is not any of the following:

0
blank
” (Empty String)
1/1/1900

When most of us look at a Where clause, we only consider two options – True and False. However, SQL Server uses three value logic, True, False, and Unknown. Since Null represents an Unknown value any comparison to it results in Unknown, not True or False. Consider the following pseudo-code:

Select * from Employees where Salary > 500

If the Salary field doesn’t allow nulls then it’s very easy to return the list of Employees which earn more than $500 per some time period. However, if Salary is a Nullable field, then any Employee without a Salary specification will not be returned. How could it be? You simply do not know whether they make more than $500 or not.

I find the easiest way for me to learn these concepts is to look at examples. Let’s start with the following base query from the Kung Fu Database.

SELECT OD.OrderID, OD.[LineNo], OD.ProductID, OD.ItemDescription, OD.OrderQty, OD.Discount, OD.ListPrice
FROM   dbo.OrderDetails OD

Null values are hightlighted.

Returns: 6 Records. This is the total data set.

Now consider the following:

SELECT OD.OrderID, OD.[LineNo], OD.ProductID, OD.ItemDescription, OD.OrderQty, OD.Discount, OD.ListPrice
FROM   dbo.OrderDetails OD
 
WHERE  OD.OrderQty >= 1

Returns: 4 Records. Those with Null Values for OrderQty are ignored.

SELECT OD.OrderID, OD.[LineNo], OD.ProductID, OD.ItemDescription, OD.OrderQty, OD.Discount, OD.ListPrice
FROM   dbo.OrderDetails OD
 
WHERE  OD.OrderQty < 1

Returns: 0 Records. Comparing any value to Null results in Unknown and those those records are not returned.

SELECT OD.OrderID, OD.[LineNo], OD.ProductID, OD.ItemDescription, OD.OrderQty, OD.Discount, OD.ListPrice
FROM   dbo.OrderDetails OD
 
WHERE  OD.OrderQty = NULL

Well, logically this should return 2 records, right? After all, you are asking for those records with Null values for OrderQty. However, the correct answer is 0 records. The reason is simple. Comparing any value to Null (even another Null) returns Unknown. This may seem confusing, but logically it makes sense. Consider my Salary example above. If Bob and Dave both have Null Salary values, do they earn the same amount? Obviously, there’s no way to know. Therefore, the query evaluates to Unknown for those records and they aren’t returned. So, how can we return the Null records?

SELECT OD.OrderID, OD.[LineNo], OD.ProductID, OD.ItemDescription, OD.OrderQty, OD.Discount, OD.ListPrice
FROM   dbo.OrderDetails OD
 
WHERE  OD.OrderQty IS NULL

Returns: 2 Records. For the same reason, we could have used “IS NOT NULL” and 4 records would return.

So, what if we want to return any records with a quantity less than 1 and include Null Values? Well, you could write that query like this:

SELECT OD.OrderID, OD.[LineNo], OD.ProductID, OD.ItemDescription, OD.OrderQty, OD.Discount, OD.ListPrice
FROM   dbo.OrderDetails OD
 
WHERE  OD.OrderQty < 1 
OR OD.OrderQty IS NULL

Returns: 2 Records.

Replacing Null Values with ISNULL or COALESCE

So that works, but it’s wordy. How can we more easily deal with nulls? Well, we can replace them with an actual value by using COALESCE or ISNULL. Each function returns the first Non-Null value found in the list of arguments. For example, SELECT ISNULL(NULL, 25) returns 25. However, SELECT ISNULL(NULL, NULL) returns Null.

Some people prefer IsNUll because the name of the function is more understable to them and it has been shown to have a very slight performance advantage. However, I use Coalesce because IsNull is limited to 2 arguments, whereas COALESE doesn’t have any realistic limit. SELECT COALESCE(NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, 7) still returns 7. Remember that it returns the first non-null value. Therefore, SELECT COALESCE(NULL, 3, NULL, NULL, NULL, 7) only returns the first value which is 3.

So, you may use Coalesce to simply these comparisons as well as for aesthetic reasons. Consider the previous Kung Fu Database query re-written with Coalesce. I’ve formatted it vertically to emphasize the difference.

SELECT
  OD.OrderID
 ,OD.[LineNo]
 ,OD.ProductID
 ,OD.ItemDescription
 ,COALESCE(OD.OrderQty,0) OrderQty -- COALESCE AND ALIASED
 ,OD.Discount
 ,OD.ListPrice
FROM
  dbo.OrderDetails OD
WHERE
  COALESCE(OD.OrderQty,0) < 1

Who determines whether or not you should replace the field like this? For the most part, the business user. So, if the business user provides the proper substitution values, the query may be written like this:

SELECT
  OD.OrderID
 ,OD.[LineNo]
 ,COALESCE(OD.ProductID, 0) ProductID
 ,COALESCE(OD.ItemDescription, 'None') ItemDescription
 ,COALESCE(OD.OrderQty, 0) OrderQty
 ,COALESCE(OD.Discount, 0) Discount
 ,COALESCE(OD.ListPrice, 0) ListPrice
FROM dbo.OrderDetails OD

Joining on NULL-able Columns

So, I’ve demonstrated how Null values can cause problems in the Where clause. For the same reasons, joining on Null-able columns is problematic as well. Consider my example Order Details table which allows a Null value for ProductID. Consider the following script:

SELECT
  OD.OrderID
 ,OD.[LineNo]
 ,OD.ProductID
 ,P.ProductID
 ,P.ProdName
 ,OD.ItemDescription
 ,OD.OrderQty
 ,OD.Discount
 ,OD.ListPrice
FROM
    dbo.OrderDetails OD
  FULL OUTER JOIN
    dbo.Products P
  ON P.ProductID = OD.ProductID

If that were written as your standard inner join, the recordset would lose the indicated record.

Grouping, Sorting, and Aggregates

In regards to Group By and Order By, Null values are considered equal. Meaning that all of the Null values are treated as one entity and are grouped and ordered as such.

Consider the following queries and result sets:

SELECT OD.OrderQty
FROM   dbo.OrderDetails OD
 
ORDER BY OD.OrderQty

SELECT   OD.OrderQty
FROM     dbo.OrderDetails OD
 
GROUP BY OD.OrderQty

Aggregate Functions
Aggregate functions such as COUNT, SUM, and AVG ignore Null values. However, one can use COUNT(*) which will count all records. Consider the following query:

SELECT
  SUM(OD.OrderQty) QtySum
  ,Avg(cast(OD.OrderQty as decimal (4,2))) NullAvg
  ,Avg(cast(COALESCE(OD.OrderQty,0) as decimal (4,2))) ZeroAvg 
 ,COUNT(OD.OrderQty) QtyCount
 ,COUNT(*) TotalCount
FROM
  dbo.OrderDetails OD

I had to Cast the quantities as decimals, because otherwise the values would have been rounded to 1. If you do the math, you’ll see that aggregate functions only considered the four records with non-null values. This is particularly noticeable when you compare the NullAvg and the ZeroAvg. The ZeroAvg actually divides the total by 7 instead of 4.

In this article, I’ve attempted through the use of many examples to describe the issues when working with Null values and how to deal with them. In the next article, I’ll discuss how to deal with blank values in Joins, Where clauses, and when creating parameter lists for SQL Server Reporting Services.

Ooh! Ooh! Pick Me! SQL Rally

David — Mon, 31 Jan 2011 19:36:12 +0000

Pick Me Mr. Kotter!

SQL Rally is a brand new event for this year. It’s kind of a hybrid between the PASS Summit and a SQL Saturday. It’s fairly low cost and being held May 11-13 in Orlando, Florida.

Anyway, the reason for this post is completely self serving. I really want to present at this event. Please register to vote and vote for my sessions. Mine are in the 10th section in the SSRS category. The abstracts I submitted are:

Title: Choose Your Own (Reporting) Adventure – SSRS Survivor Series
Abstract: Create a story of guided analytics using Reporting Services drill down, drill through, and expressions. Sure, you may be using drill down already, but I’ll show you how to make the most of Reporting Services interactions, and more importantly how to avoid becoming dragon chow at the end.

Title: The Power of Parameters – SSRS Survivor Series
Abstract: Want to know the best techniques to help your user narrow their search in the most efficient ways possible? Come watch me demonstrate which parameter techniques are best for certain situations and field types. Real world problems and applications are stressed, and I’ll demonstrate why some commonly accepted methods simply do not work and should be avoided.

–My name is Made2Mentor, and I approved this message.

SQL Saturday Dallas #56 Re-Cap

David — Thu, 28 Oct 2010 14:00:55 +0000

Ryan, Vic, and yours truly.

SQL Saturday #56 BI Edition went off without a hitch last Saturday. Those who read this blog, probably remember the awesome experience I had at my first SQL Saturday.

Once again, everything went wonderfully. Our event was scaled back this time which translated into fewer hassles. Microsoft and Artis Consulting were our only sponsors. Microsoft provided the building, took care of security, check in, and all of those little things that need to be handled the day of the event, while Artis provided food for breakfast and lunch. This left many of us on the planning committee with more time to enjoy the day with fewer things to worry about.

Leading people astray....

I was asked to present two topics this time:

Creating Interactive Dashboards with PowerPivot. Slicing Your Way to Data Nirvana
Who, What, Where. The Power of SSRS Parameters

My presentations went very well, and were well received. My evaluations were great, but I do know that I need more practice. That shouldn’t be too hard to get as I thoroughly enjoy it anyway.

I wanted to take a moment to thank a few people. I want to thank the folks on the volunteer committee and NTSSUG board. Folks like Sri Sridharan, Sean and Jen McCown, Tim Mitchell, Vic Prabhu, Ryan Adams, and Ganesh Gopal. I’d also like to thank Rakesh Kanjani (Rocky) for taking pictures for the event and Andy Eggers who works harder on event day than any other three people I know.

Let me let you in on a little tip. Go to the after party. They’re not crowded, though they should be. Yes, I know you’re tired after a full day of learning and networking, but go anyway. The best conversations, both technical and non-technical, can be had there. People are ready to relax, open up, and the atmosphere is great for networking.

A note to those who attended my sessions at SQL Saturday, and the week prior at Consona Connect 2010: I haven’t forgotten that I owe both groups a few blog posts going into more detail on the topics I presented. I promise to have those out by the end of next week.

I’ll also be blogging about M-Data Analytics as well as a posts regarding PASS Summit which I’ll be attending for the first time this year.

I’m so thrilled to be going, and you should go too. Tomorrow, I’m going to tell you why.

Stay Tuned!

SQL Server Maintenance Plans for Small/Mid-sized Businesses FAQs

David — Tue, 31 Aug 2010 14:00:06 +0000

I get a ton of questions and calls about SQL Server Maintenance issues from Administrators who just want their server to work and be protected. They’re not interested in the intricate structure of a transaction log, how to use wait stats, etc. While many database administrators who read this article will disagree or perhaps outright flame me, (cough Sean McCown cough ) I feel that SQL maintenance plans are a very effective means for non-dbas to protect their data.

Rather than recreate tutorials that others have already written and take the same screenshots they provide, I’ll link you to them and suggest settings that are applicable to most M2M companies. Keep in mind that every situation is different, what I’m offering are very general guidelines.

Maintenance Plan Tutorials

The simplest way to maintain a SQL Server is to use the Maintenance Plan Wizard. Please select a link to instructions for your server version. Each site has an easy to follow tutorial. These types of tutorials will explain the purpose of most settings, but will not make recommendations because so many factors come into play.

SQL Server 2000
SQL Server 2005 Part One, two, and three.
SQL Server 2008

Database Mail

Before moving on to the guidelines, make sure you also have database mail set up for notifications. A Maintenance Plan isn’t any good if it isn’t running successfully. Unless you want to manually check the server each day, you’ll need to set up notifications.

For SQL Server 2000, there are a couple of ways to do this neither of which is particularly appetizing. The easiest way is to use Outlook on your server. However, ideally you shouldn’t be running anything but SQL Server on your SQL Server (more on that in a future article). You can send mail using stored procedures using SQL Server but I don’t expect that many of you would want to deal with that. It’s a mess and also a perfect excuse to nag your boss to approve a SQL Server upgrade.

SQL Server 2005 and 2008 have mail capability built in and it’s very easy to set up. Visit the following sites for complete walkthroughs:

FAQs and Guidelines for Small/Mid-sized Companies

Alright, now that the groundwork is out of the way, my recommendations follow. Companies which use M2M tend to be small to mid-sized. Most M2M databases are between 5 and 100GB and with data smaller databases like these, performance isn’t a huge issue. These guidelines assume that you fit this profile. The following questions are frequently asked of me, and most of them are business related, rather than technical.

Which recovery model should I use?
I respond to this question with a question. “How much data can you afford to lose?” If you are willing to risk losing a day’s productivity, then Simple Recovery Model is fine and you won’t have to worry about transaction log backups. However, if you need better protection than that, use the Full Recovery Model. I have some databases set to the Simple Recovery Model because I simply don’t care about them. One example is WebSense. Would it be the end of the world if I lost 24 hours employee web surfing records? Not really. I wrote a more detailed explanation of recovery models here.

How often should I do transaction log backups?
Once again, how much work can you afford to lose? Most of the M2M sites I’ve dealt with which use the Full Recovery Model, schedule transaction log backups every 15 to 60 minutes. Keep in mind that whatever time span you set for this represents the maximum amount of work you can lose. Get your backups (full and t-log) off the server as quickly as possible. I’ve looked at more than a few M2M Servers where the backups were kept on the same drive volume as the database. This is a waste of time.

How long should I keep backups?
I hate to keep echoing questions back at you, but how long do you need them? A couple of the companies I do business with keep their full database backups almost a year. However, they move them to tape every day and only have one or two copies internally. Read further about backup options for details.

Do I have to worry about SQL Database Integrity?
I often receive blank looks when I mention integrity checks. M2M recommends moving from Visual FoxPro to SQL Server because it is more stable. However, that doesn’t mean you can’t get data corruption. I recommend checking database integrity before your nightly full backup. If any database fails an integrity check contact a qualified database administrator to deal with it. Unlike the old days of re-indexing, purifying, and packing Visual FoxPro tables, SQL database integrity failures can be a serious problem. If you decide to try to handle that yourself, look for articles written by known experts such as the great Paul Randal. While we’re on the subject of data integrity, make sure to check that the maintenance plan verifes the integrity of the backups as well. However, you still need to perform test restores to be absolutely sure that your backups are sound.

Do I need to re-index my SQL Databases?

Once again, when I answer this question I am assuming that you have a database between 5 and 100GB and have decent server hardware as well. The answer is yes, you should index your SQL Databases. I index mine on weekends, but I could just as soon index it on weeknights as it doesn’t take much time with a database that small. You won’t notice much a difference in performance after indexing either as M2M is not heavily indexed. Since time isn’t a factor here, set your indexes to rebuild the indexes which effectively drops and then recreates them. Also, go ahead and update statistics as well.

Can I shrink my database and make it smaller?
No, do not ever shrink your database unless you have a compelling reason to do so. Unless your database grew for a specific reason do not shrink your database because it will then have to grow again. These processes are expensive and unnecessary. A typical scenario which may require a shrink would be an over-sized transaction log which went without a backup for a long time. You can purchase that shirt here.

How often should I perform a test restore?
Do you have a good backup? Are you sure??? You know that scene in My Cousin Vinny where Vinny and Mona Lisa are arguing about the dripping faucet?

Mona Lisa: Did it ever occur to you that it could be turned off and drip at the same time?
Vinny: No, because if you turned it off, it wouldn’t drip.
Mona Lisa: Maybe it’s broken.
Vinny: Is that what you’re saying? It’s broken?
Mona Lisa: Yeah, that’s it; it’s broken.
Vinny: You sure?
Mona Lisa: I’m positive.
Vinny: Maybe you didn’t twist it hard enough.
Mona Lisa: I twisted it just right.
Vinny: How can you be so sure?
Mona Lisa: If you will look in the manual, you will see that this particular model faucet requires a range of 10-16 foot pounds of torque. I routinely twist the maximum allowable torquage.
Vinny: How can you be sure you used 16 foot pounds of torque?
Mona Lisa: Because I used a Craftsman model 1019 Laboratory edition, signature series torque wrench. The kind used by Cal Tech High Energy physicists, and NASA engineers.
Vinny: In that case, how can you be sure that’s accurate?
Mona Lisa: Because a split second before the torque wrench was applied to the faucet handle, it had been calibrated by top members of the state and federal Departments of Weights and Measures, to be dead-on balls accurate. Here’s the certificate of validation.
Vinny: “Dead-on balls accurate”?
Mona Lisa: It’s an industry term.

The scene is funny, but the point is valid. How can you be sure that your backups are “Dead-on balls accurate?” Perform test restores. If you haven’t performed a test restore then the backup cannot be relied upon. I can hear the question forming in your mind, “Do I have to perform a test restore every single day?” No, I don’t expect you to check every single backup to verify that it restores. However, once or twice a month isn’t out of line. After all, with the size of most of our databases, it only takes 10 to 15 minutes to perform a test restore anyway.

In future articles, I’ll show you how to perform database restores using both full and transaction log backups. I’ll also document the system I’m currently using. I backup both a SQL 2000 and 2008 R2 database server using t-sql scripts, do-it-yourself compression and encryption, as well as nightly test restores. It will take me awhile to document, since I have to re-set all of that up from scratch at home. I’ll try to post these articles before Consona Connect.

Any questions? Did I forget anything? Any disagreements?

Using User Defined Functions to Pre-Format Fields

David — Thu, 24 Jun 2010 15:00:22 +0000

Some DBAs would cringe at what I’m about to tell you, but I often use T-SQL to format my data prior to presenting it in a report. Many experts feel that data formatting should always be done in the presentation layer. There are a couple of reasons why I do this.

I re-use as much code as I can. I create base templates for most of the main modules in M2M (Sales, Purchasing, Accts Payable, Etc) and these templates include the formatting code. This results in a professional, standardized image to my reports.
I report using many different programs as my presentation layer and I want to re-use my scripts for all of them.

Consider the following script:

Select fsono, fenumber, frelease, fpartno from m2mdata50..sorels

The first few rows of data from the M2M educational database return:

000001	001	000	CP2080 
000001	002	000	FF204W 
000002	001	000	WF201R 
000002	001	001	WF201R 
000002	001	002	WF201R 
000002	001	003	WF201R 
000002	001	004	WF201R

Do you notice the distraction of the leading zeros? When searching for a sales order number, does anyone really enter “000003″? These are a problem in most of the modules of M2M. As I mentioned before, many DBAs feel that you should remove them via formatting in the presentation program of your choice. However, I regularly create reports in SQL Reporting Services, Crystal Reports, Visual FoxPro, and Excel. This means that if I don’t pre-format with T-SQL, I will have to adapt the presentation later manually, which and the technique will be different for each reporting media. This is time consuming and error prone.

As always, please read my standard disclaimer before implementing any code from my site. Using T-SQL, you could strip the preceding zeros like this:

select substring(fsono, patindex('%[^0]%',fsono), 10) SONo,
substring(fenumber, patindex('%[^0]%',fenumber), 10) SOItemNo,
substring(frelease, patindex('%[^0]%',frelease), 10) Release,
fpartno PartNo from m2mdata50..sorels

Which yields this:

1	1	000	CP2080 
1	2	000	FF204W 
2	1	000	WF201R 
2	1	1	WF201R 
2	1	2	WF201R 
2	1	3	WF201R 
2	1	4	WF201R

That works, but it’s too wordy for me. A better option is to use a User Defined Function (UDF) to clean up the data. I’ve demonstrated how to write these in the past.

Consider the following:

CREATE FUNCTION RemoveZeros(@ZeroField VARCHAR(25))
  RETURNS VARCHAR(12)
AS
  BEGIN
    DECLARE @ZerosRemoved   VARCHAR(12)
    DECLARE @Length   INT
    SET @length = len(@ZeroField)
    SELECT @ZerosRemoved =
             CASE
               When @ZeroField = '000'
               then
                 'M'
               WHEN @Length > 0
               THEN
                 substring(
                   @ZeroField,
                   patindex(
                     '%[^0]%',
                     @ZeroField),
                   @Length)
               ELSE
                 ''
             END
    RETURN @ZerosRemoved
  END

Which produces output like this:

1	1	M	CP2080 
1	2	M	FF204W 
2	1	M	WF201R 
2	1	1	WF201R 
2	1	2	WF201R 
2	1	3	WF201R 
2	1	4	WF201R

Can you see the possibilities? You can use this UDF in other modules such as shipping and invoicing as well. It works particularly well in shipping because the item numbers are 6 digits. Have any of you ever had a shipper with a million line items on it?

Select fshipno, fitemno, fpartno from m2mdata50..shitem
 
000001	000001	CP2080 
000002	000001	FF204W 
000003	000001	L20100 
000003	000002	L20200 
000004	000001	EF120100P

Becomes:

Select dbo.removezeros(fshipno) ShipNo,
       dbo.removezeros(fitemno) ItemNo,
       fpartno PartNo
from m2mdata50..shitem
 
1	1	CP2080 
2	1	FF204W 
3	1	L20100 
3	2	L20200 
4	1	EF120100P

The performance impact is negligible, and the both the script and the output are well formatted.

What other UDF’s do you think would be useful for M2M reporting?

Five Reasons You Should Volunteer With Your Local SQL User Group and Host a SQL Saturday

David — Wed, 19 May 2010 13:00:25 +0000

I took the plunge this year and started volunteering for the North Texas SQL Server User Group. Originally I started attending because I wanted to learn more about SQL Server, but it didn’t take long before I realized that there was so much more to be gained from being there.

Why should you volunteer with your local user group?

You get to meet really cool people.

I realize that I don’t know most of you personally, but I think most of you are like me. The reason you read this blog is because you want to know more about your profession than the average person. That’s likely how you found me in the first place. The folks at your user group are just like us. They are passionate about their craft, fascinated by technology, and want to be the best they can be. However after you get to know them better, you’ll also find that many of them are caring and really fun to be around.

I can hear you saying, “Yeah David, but I can get all of that by going to the monthly meetings, why should I bother to volunteer?” I attended this user group for almost 2 years before I volunteered and had no idea who these people really were. I had talked to them of course, but you only get to know them as you work WITH them. You become a member of the club, feel like you belong, and they let their hair down.

You can network with some of the best minds in SQL Server.

Some of the best minds in SQL Server present at SQL Saturdays. They fly in at their own expense to share their knowledge and network with you. These are the people who write the books on SQL Server, MVPs and other luminaries. When you volunteer, you can get exclusive access to them.

Also, remember those smart, motivated folks in your local area that I mentioned above? On the volunteer committee I’m working side by side with two Microsoft MVPs and more folks who are on their way to becoming MVPs. I’m spending weekends volunteering with them and getting to know them well. These are the folks who can help you find your next job.

Do you want to eventually give SQL presentations? You should start by volunteering.

When I chose the name Made2Mentor, it wasn’t without a great deal of thought. I have always taught others and mentored them in something that I was good at. I’m learning SQL Server rapidly and intend to begin presenting on various aspects of it.

If you’ve already volunteered, and I intend to continue to do so, approaching the leader of the User Group is an easy matter because you’ve already worked extensively with him. I may even present a session at the next SQL Saturday.

You can learn so much from the other volunteers, maybe even find a mentor or two.

I’ve already received a ton of help from people on the volunteer committee that I would not have been able to get had I not volunteered. While I know you can get basic help from forums or Google searches, you get so much more from people with whom you’ve built a relationship.

Once you get to know them, they’ll take the time to look at your problem personally and guide you in the right direction. It’s not just technical help either. Many of us are very bright technically, but lack the soft skills. Many of the people who run these groups have already mastered the soft skills of interpersonal communication, and you can learn from them. It is only natural that they eventually become your mentors.

Volunteering makes you feel good about yourself.

I know that I often focus on the career aspects of networking, but it’s not all about your personal gain. It feels good to give to the community, to help the other volunteers without worrying about what you’ll get from it. There’s a real sense of achievement which comes from hosting an event like SQL Saturday and knowing that you had a part in it.

So, I encourage you to join your local SQL Server User Group and volunteer. If you have the opportunity to volunteer for a project such as a SQL Saturday, jump in and do your best. There are so many benefits, that you’ll find it’s totally worth it.

I’d like to my fellow members of the planning committee Sri Sridharan, Sean and Jen McCown, Tim Mitchell, Vic Prabhu, Ryan Adams, as well as the rest of the volunteers for SQL Saturday for making the experience great.

By the way, if any of you are coming to SQL Saturday Dallas, then look me up. I’ll be running all over the place, helping to get things done.

PS. To my regular readers, I apologize about my lack of blogging lately. Obviously I’ve been busy with SQL Saturday and I’ve had a great deal of pressure with my work the past few weeks. I’ll be back on track and will continue with the disaster recovery theme next week.

BI Ethics

David — Wed, 09 Dec 2009 18:46:49 +0000

The ethics of business intelligence is a touchy subject that many people don’t want to talk about. However, more attention in the past several years has been given to this subject mostly due to ethics scandals at huge companies such as Enron, Tyco, and WorldCom. The Sarbanes-Oxley Act came about because of such scandals and the billions of dollars they cost investors as well as the decrease in investor confidence in the stock markets, but it only applies to publicly held companies.

This is all well documented, and related information can be found all over the web. Recently a Code of Conduct for DBAs was proposed by SQL DBA, Brad McGehee. This is an excellent, albeit broad set of guidelines for DBA behavior.

My last article discusses a relatively benign situation where a decision was made how to calculate and provide statistics. In this article, I’d like to focus on business intelligence. What do you as a Business Intelligence Expert do when asked to provide questionable information? I’ll give the following example to facilitate discussion.

The Accounting Manager of your organization requests a report which details Accounts Receivable Aging. You code the SQL for the report creating groups of invoices that are 30 days, 60 days, 90 days, and more past due. You include an attractive chart and send the report to the Manager for approval. He reviews it and confides in you that his new bonus structure (or perhaps even his job) is dependent upon these numbers. He requests that since ACME Rockets is always late, Wile E. Coyote is always out in the desert chasing the Road Runner instead of paying bills, that they should be removed from the report. His reasoning is that no matter how well the manager performs his job, ACME will never pay in less than 90 days and this drastically lowers his rating.

Think about that for a second…. I’ll wait.

Now let’s suggest that the Accounting Manager is your direct supervisor. You may actually lose your job if you refuse. Look deep within yourself, did the answer change?

Anyway, the following poll is anonymous. What would you do? Feel free to explain why you answered the way you did in the comments.
Note: There is a poll embedded within this post, please visit the site to participate in this post's poll.