Tuesday, April 5, 2011

What About "Under-Used" Indexes?

We've all spent time looking for unused indexes (right?), but today my boss asked about under-used indexes. Now, I wrote a script a while ago to find completely unused indexes, but finding indexes that were only being used a little was just different enough to require a new script.

The script below displays all indexes, with the least-used (including unused) ones first, and totals for user and system reads and updates. I figured having the ratio between reads and updates might be useful, since if it takes more updates to maintain the index than it gets used, you might want to investigate its cost-benefit.

Some caveats:

  • If your server just rebooted, these numbers probably won't mean a lot; wait for at least a full day to allow time for the indexes to be read or written to.
  • Also, don't forget about periodic events like end-of-month processing. You don't want a phone call at 12:15 am on the first of the month when it turns out that index was needed after all.
  • Just because it costs more to maintain an index doesn't mean it's not still worth having. The cost of maintaining the index is probably spread out over a long period of time; the benefit to a waiting user of a query that takes two seconds instead of five minutes to run might outweigh that cost. It depends on your particular environment.
  • Note that indexes that support primary key constraints are excluded, as I'm pretty sure I'm not going to ever want to drop them!

Finally, consider disabling any under-used indexes you find, rather than just dropping them. If it turns out the index was important, you'll still have to rebuild the index, but at least the DDL code will be there already.

-- Display unused and under-used indexes. 

DECLARE @dbid INT = DB_ID(DB_NAME())

; WITH cteUser AS
(
    SELECT object_id,
           index_id,
           user_seeks + user_scans + user_lookups   AS 'User Reads',
           user_updates                             AS 'User Updates'
      FROM sys.dm_db_index_usage_stats   
     WHERE database_id = @dbid
)
,
cteSystem AS
(
    SELECT object_id,
           index_id,
           system_seeks + system_scans + system_lookups   AS 'System Reads',
           system_updates                                 AS 'System Updates'
      FROM sys.dm_db_index_usage_stats   
     WHERE database_id = @dbid
)
,
cteTotal AS
(
    SELECT u.object_id,
           u.index_id,
           [User Reads]   + [System Reads]     AS 'Total Reads',
           [User Updates] + [System Updates]   AS 'Total Updates'
      FROM cteUser     u
      JOIN cteSystem   s
        ON u.object_id = s.object_id
       AND u.index_id  = s.index_id
)
,
cteReadToUpdateRatio AS
(
    SELECT object_id,
           index_id,
           CONVERT(NVARCHAR,
                  CONVERT(MONEY, ISNULL(
                                        CAST([Total Reads] AS REAL)
                                        /
                                        NULLIF([Total Updates], 0.0)
                                        , [Total Reads]
                                        )
                               , 1
                         )
                  )   AS 'Read-to-Update Ratio'
                  
      FROM cteTotal
)
SELECT OBJECT_NAME(i.object_id)               AS 'Table Name',
       i.name                                 AS 'Index Name',
       u.[User Reads],
       u.[User Updates],
       s.[System Reads],
       s.[System Updates],
       t.[Total Reads],
       t.[Total Updates],
       r.[Read-to-Update Ratio],
       t.[Total Reads] + t.[Total Updates]    AS 'TOTAL READS + UPDATES'
  FROM cteUser                        u
  JOIN cteSystem                      s
    ON u.object_id = s.object_id
   AND u.index_id  = s.index_id
  JOIN cteTotal                       t
    ON u.object_id = t.object_id
   AND u.index_id  = t.index_id
  JOIN cteReadToUpdateRatio           r
    ON u.object_id = r.object_id
   AND u.index_id  = r.index_id
  JOIN sys.indexes                    i
    ON u.object_id = i.object_id
   AND u.index_id  = i.index_id
  JOIN sys.objects                    o
    ON u.object_id = o.object_id
 WHERE OBJECTPROPERTY(o.object_id, 'IsUserTable') = 1
   AND i.is_primary_key = 0
   --AND (t.[Total Reads] = 0                                  -- These definitely need to be dropped. 
   -- OR CAST(r.[Read-to-Update Ratio] AS REAL) < 1.0)         -- These may need to be dropped. 
 ORDER BY CAST(r.[Read-to-Update Ratio] AS REAL)

Friday, April 1, 2011

ISNULL or COALESCE? Sometimes it Really Does Matter

After seeing some poor performance in a query that I couldn't figure out, I ended up reading a lot of discussions about the "ISNULL / COALESCE" debate. Some people feel one is faster than the other, and some people like COALESCE because it's ANSI standard. (And they hold these opinions very strongly.)

I always thought the two were identical (when just two values are involved, of course), but it turns out that they have a difference that can affect performance. The value that ISNULL returns is typed to be same as the type of the first argument. The value that COALESCE returns is typed to be the same as the argument with the highest data type precedence.

Adam Machanic says: "What does this have to do with query performance? Sometimes, when using ISNULL or COALESCE as part of a predicate, a user may end up with a data type mismatch that is not implicitly convertable and which therefore causes a table scan or other less-than-ideal access method to be used."

Data type mismatches can cause table scans? I had totally forgotten about that. Makes sense though: if there's no implicit conversion, you'll have to use a CAST or CONVERT, which means each row will have to be evaluated.

That's what was causing the poor performance I was seeing. I wouldn't advocate a global search-and-replace, but I will be looking more closely at the COALESCE statements I come across.

Monday, February 7, 2011

Monitor the Progress of Long-Running Events

I never knew you could monitor the progress of long-running processes like DBCC CHECKDB, backups, shrinking files, rebuilding indexes, etc. until I read an offhand comment by Kalen Delaney (which I can't find now).   Now I keep this script open in a SSMS window about 90% of my day!

-- Displays the progress of several kinds of commands.  See BOL.
; with cte1 as
(
    select command                              as command,
           percent_complete   /  100.0          as percent_complete, 
           total_elapsed_time / 1000.0 / 60.0   as elapsed_minutes
      from sys.dm_exec_requests
     where percent_complete > 0.0
)
, cte2 as
(
    select command                                                               as command,
           cast(percent_complete * 100.0 as float(6))                            as percent_complete,
           cast(elapsed_minutes as int)                                          as elapsed_minutes,
           cast((elapsed_minutes / percent_complete) - elapsed_minutes as int)   as remaining_minutes
      from cte1
)
select command                                                           as 'Command',
       percent_complete                                                  as '% Complete', 
       elapsed_minutes                                                   as 'Elap Min',
       remaining_minutes                                                 as 'Left Mins',
       cast(dateadd(minute, remaining_minutes, getdate()) as nvarchar)   as 'ETA'
  from cte2
 order by percent_complete desc

The output looks like this:

Command             % Complete    Elap Min    Left Mins    ETA
DbccFilesCompact      31.62873          15           33    Feb  6 2011 11:50PM

Monday, January 31, 2011

Removing All Duplicate Tabs, Linefeeds, Returns, and Spaces

The trace tables created by SQL Profiler are nice and all, but it's a pain to search through the TextData column.  The code below uses a brute force method to replace all contiguous whitespace characters with a single space.  This makes it possible to search the TextData column without differences in whitespace getting in the way.

    -- Removing All Duplicate Tabs, Linefeeds, Returns, and Spaces. 

    DECLARE @RowsAffected INT = 0
    SET NOCOUNT ON
    
    -- Create all 16 (4^2) Tab, Linefeed, Return, and Space combinations.
    DECLARE @PairTT VARCHAR(2) = CHAR( 9) + CHAR( 9)    
    DECLARE @PairTL VARCHAR(2) = CHAR( 9) + CHAR(10)    
    DECLARE @PairTR VARCHAR(2) = CHAR( 9) + CHAR(13)    
    DECLARE @PairTS VARCHAR(2) = CHAR( 9) + CHAR(32)    
    DECLARE @PairLT VARCHAR(2) = CHAR(10) + CHAR( 9)    
    DECLARE @PairLL VARCHAR(2) = CHAR(10) + CHAR(10)    
    DECLARE @PairLR VARCHAR(2) = CHAR(10) + CHAR(13)    
    DECLARE @PairLS VARCHAR(2) = CHAR(10) + CHAR(32)    
    DECLARE @PairRT VARCHAR(2) = CHAR(13) + CHAR( 9)
    DECLARE @PairRL VARCHAR(2) = CHAR(13) + CHAR(10)
    DECLARE @PairRR VARCHAR(2) = CHAR(13) + CHAR(13)
    DECLARE @PairRS VARCHAR(2) = CHAR(13) + CHAR(32)
    DECLARE @PairST VARCHAR(2) = CHAR(32) + CHAR( 9)
    DECLARE @PairSL VARCHAR(2) = CHAR(32) + CHAR(10)
    DECLARE @PairSR VARCHAR(2) = CHAR(32) + CHAR(13)
    DECLARE @PairSS VARCHAR(2) = CHAR(32) + CHAR(32)
                                                    
    WHILE 1=1
    BEGIN
        SELECT @RowsAffected = COUNT(*) 
          FROM dbo.TraceTableReportPermanent
         WHERE TextData LIKE '%' + @PairTT + '%'
            OR TextData LIKE '%' + @PairTL + '%'
            OR TextData LIKE '%' + @PairTR + '%'
            OR TextData LIKE '%' + @PairTS + '%'
            OR TextData LIKE '%' + @PairLT + '%'
            OR TextData LIKE '%' + @PairLL + '%'
            OR TextData LIKE '%' + @PairLR + '%'
            OR TextData LIKE '%' + @PairLS + '%'
            OR TextData LIKE '%' + @PairRT + '%'
            OR TextData LIKE '%' + @PairRL + '%'
            OR TextData LIKE '%' + @PairRR + '%'
            OR TextData LIKE '%' + @PairRS + '%'
            OR TextData LIKE '%' + @PairST + '%'
            OR TextData LIKE '%' + @PairSL + '%'
            OR TextData LIKE '%' + @PairSR + '%'
            OR TextData LIKE '%' + @PairSS + '%'

        IF @RowsAffected > 0
            UPDATE dbo.TraceTableReportPermanent
               SET TextData = REPLACE(
                              REPLACE(
                              REPLACE(
                              REPLACE(
                              REPLACE(
                              REPLACE(
                              REPLACE(
                              REPLACE(
                              REPLACE(
                              REPLACE(
                              REPLACE(
                              REPLACE(
                              REPLACE(
                              REPLACE(
                              REPLACE(
                              REPLACE(TextData, @PairTT, CHAR(32)),
                                                @PairTL, CHAR(32)), 
                                                @PairTR, CHAR(32)), 
                                                @PairTS, CHAR(32)),
                                                @PairLT, CHAR(32)),
                                                @PairLL, CHAR(32)),
                                                @PairLR, CHAR(32)),
                                                @PairLS, CHAR(32)),
                                                @PairRT, CHAR(32)),
                                                @PairRL, CHAR(32)),
                                                @PairRR, CHAR(32)),
                                                @PairRS, CHAR(32)),
                                                @PairST, CHAR(32)),
                                                @PairSL, CHAR(32)),
                                                @PairSR, CHAR(32)),
                                                @PairSS, CHAR(32))
             WHERE TextData LIKE '%' + @PairTT + '%'
                OR TextData LIKE '%' + @PairTL + '%'
                OR TextData LIKE '%' + @PairTR + '%'
                OR TextData LIKE '%' + @PairTS + '%'
                OR TextData LIKE '%' + @PairLT + '%'
                OR TextData LIKE '%' + @PairLL + '%'
                OR TextData LIKE '%' + @PairLR + '%'
                OR TextData LIKE '%' + @PairLS + '%'
                OR TextData LIKE '%' + @PairRT + '%'
                OR TextData LIKE '%' + @PairRL + '%'
                OR TextData LIKE '%' + @PairRR + '%'
                OR TextData LIKE '%' + @PairRS + '%'
                OR TextData LIKE '%' + @PairST + '%'
                OR TextData LIKE '%' + @PairSL + '%'
                OR TextData LIKE '%' + @PairSR + '%'
                OR TextData LIKE '%' + @PairSS + '%'
        ELSE
            BREAK
             
        RAISERROR('Deleted %d whitespace pairs from dbo.TraceTable',
                   10, 1, @RowsAffected) WITH NOWAIT
    END